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IMPROVING STUDENT ATTAINMENT IN COMMUNITY COLLEGES: 
INSTITUTIONAL CHARACTERISTICS AND POLICIES 

Executive Summary 

Community colleges are a crucial point of access to higher education for low-income, 
minority, and other underserved students. Indeed, these groups are overrepresented (with 
respect to their share of undergraduate enrollment) in two-year and less-than-two-year 
postsecondary institutions. The community college access mission is built on low tuition, 
convenient location, flexible scheduling, an open-door admissions policy, and programs 
and services designed to support students who may have various socio-economic and 
academic barriers inhibiting postsecondary success. If community colleges — or similar 
institutions — were not available, many of these students would not have an opportunity to 
attend higher education. 

While access to community colleges is an important first step for a wide variety of 
students, they must also be successful after they have enrolled. Unfortunately, many 
students never finish a degree. For example, only 36 percent of students who enrolled in a 
community college as their first postsecondary enrollment in the 1995-96 school year had 
completed a certificate, associate, or bachelor’s degree within six years. Low-income, 
minority, and first-generation college students all have even lower six-year completion 
rates. Although many students can benefit from a community college education even if 
they do not complete a degree or certificate, community college faculty and 
administrators would all like to see completion rates rise. 

This report is part of a broad initiative funded by the Lumina Foundation for Education. 
The initiative, Achieving the Dream: Community Colleges Count, will initially work with 
27 community colleges in five states to help them increase retention, completion, and 
success for low-income students, students of color, first-generation college students, and 
other underserved groups. 

The data and analysis in this report speak to four main goals: 

(1) to provide general descriptive information on community college student 
characteristics and educational outcomes and make comparisons with other higher 
education sectors; 

(2) to discuss the importance of college completion and transfer, reviewing the 
controversy about whether completion is a meaningful or justifiable standard for a 
community college; 

(3) to review the state of research on the determinants of student outcomes in 
community colleges in order to provide some programmatic guidance for colleges 
working towards improving their student outcomes and offer recommendations 
for improving the quality and effectiveness of the research; and 

(4) to initiate our own program of empirical research on institutional graduation rates 
by conducting an analysis of community college graduation rates and developing 
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a benchmarking system for evaluating college performance that takes account of 
differences in college characteristics. 

Community College Enrollment and Degree Completion 

To analyze student enrollment, we used data from Integrated Postsecondary Education 
Data System (IPEDS), which are collected annually by the National Center for Education 
Statistics (NCES) from all postsecondary institutions in the United States. In order to 
examine the persistence and graduation experience of minority and low-income students, 
we relied on data from the Beginning Postsecondary Students Longitudinal Study of 
1995-96 (BPS96). This survey is based on a sample of students who started college for 
the first time in the 1995-96 school year. The students in the sample were re-interviewed 
in 1998 and 2001, providing information on what happened to them six years after their 
initial enrollment. 

Enrollment 

IPEDS data indicate that in the decade following 1992, undergraduate enrollment in the 
United States grew by 1 1 percent, totaling almost 15 million students by the fall semester 
of 2002. All of that growth was accounted for by minority students, since college 
enrollment for White students actually fell by more than 600,000 students. The growth of 
Hispanic enrollment was particularly strong, increasing by over 50 percent to over 1.7 
million students. Some of this growth can be attributed to the overall growth of the 
Hispanic population in this country during the 1990s. 

Two-year public institutions (henceforth called community colleges) account for 42 
percent of total enrollments, while four-year institutions (public and not-for-profit 
together) account for 5 1 percent of total enrollments. Thus, all other categories (all less 
than two-year, two-year not-for-profit and for-profit, and four-year for-profit institutions) 
only enroll seven percent of all postsecondary students. 

Community colleges are particularly important for Hispanics, African-Americans, and 
Native Americans (included under “other”), who are all overrepresented in two-year 
institutions. In addition, lower income students and those whose parents have less 
education are underrepresented in four-year institutions and overrepresented in two-year 
and less-than-two-year colleges. Community colleges enroll the largest number of low- 
income and first-generation students, but these students account for a larger share of 
enrollments at for-profit institutions than at community colleges. 

Degree Completion and Transfer 

After six years, 36 percent of community college students in the BPS96 sample had 
completed a certificate, associate, or bachelor’s degree. Degree completion rates are 
higher in four-year institutions and, not surprisingly, a larger share of the degrees for 
those colleges are accounted for by bachelor’s degrees. 
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Within community colleges, minorities are less likely than Whites to complete a degree 
or certificate. Only 27 percent of African-Americans had received any type of award 
within six years, and most of those awards were certificates. Hispanics complete at a 
slightly higher rate, and a larger proportion of their awards are accounted for by associate 
and bachelor’s degrees. 

With respect to family income, community college students from the top half of the 
family income distribution and those with at least one parent with a bachelor’s degree are 
much more likely to earn some award — and these awards tend to be degrees rather than 
certificates. In addition, high socioeconomic status (SES) students who had not 
completed in six years were more likely to have transferred and still be enrolled at a four- 
year college. Not surprisingly, community college students with higher secondary school 
grade point averages and assessment test scores are also more likely to earn a degree. 

Earlier research on bachelor’s degree attainment demonstrating that attendance patterns 
strongly influence completion, and that delaying enrollment after high school, attending 
part time, or interrupting enrollment significantly decrease the chances that a student will 
earn a bachelor’s degree holds for community college students as well. Moreover, 
community college students are much more likely to follow these non-traditional 
attendance patterns than four-year students, so it is significant that the patterns that put 
students most at risk of failing to persist are those that characterize most community 
college students. 

Degree Completion and Transfer as Indicators of Student Success 

Many community college students benefit from the skills they learn in college, even if 
they never earn a degree. There is now, however, an increase in the emphasis on 
graduation and we conclude that increasing degree completion and transfer rates should 
be central institutional goals, although certainly other measures of student success should 
also be considered. 

Accrediting agencies, long criticized for an overemphasis on “inputs” such as the 
credentials of the faculty and number of books in the library, are beginning to focus more 
on outcomes as a result of changed accreditation policies and demands from 
policymakers for greater accountability. More than half of the states now engage in 
“performance budgeting,” under which state officials, in drafting annual budgets, take 
into account public colleges’ performance, and 18 states have performance funding 
schemes in which public colleges gain or lose set amounts of money based on how well 
they meet certain standards. Further, under the Higher Education Amendments of 1998, 
to be eligible to receive federal financial aid, colleges are already required to report 
graduation rates for cohorts of first-time, full-time students in 150 percent of the 
“traditional” graduation period (three years for community colleges and six years for 
baccalaureate-granting institutions). 

Community college educators, advancing three broad arguments, have resisted the use of 
graduation rates either as an accountability measure or as a normative goal. First, they 
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argue, with some research-based support, that many students at their institutions are not 
seeking degrees. Instead, in many cases, students enroll with the goal of learning some 
specific skills, perhaps to gain a promotion at their current job. Second, they assert that 
many factors that thwart graduation are beyond the control of the colleges, citing the fact 
that college students face serious barriers to success in college such as family and work 
responsibilities and deficient academic preparation. Third, they argue that students 
increasingly attend several colleges before completing their degrees. 

Furthermore, while institutional graduation rates can be taken as underestimates of 
individual graduation rates, there is still useful information to be gleaned from examining 
differences among colleges in institutional graduation rates. Explaining why some 
colleges with similar characteristics and similar types of students have much higher rates 
than others might offer insights into possible policies and practices that could improve 
performance in many colleges. In addition, any individual college can analyze its 
completion and persistence rate as an important outcome of programs aimed at improving 
its performance. 

Nevertheless, there are other factors that suggest that college completion or transfer 
should be important goals for community college students. First, research indicates that 
earning credits without completing a degree does have an economic value, but students 
get an additional financial benefit from the credential. Indeed, for students in academic 
majors in community college, the real benefit comes from eventually earning a bachelor’s 
degree. Earning small amounts of credit in academic subjects also has no measured 
economic value. Therefore, research does not refute the argument that short-term course 
taking to upgrade skills can be valuable for students, but neither does it provide strong 
support for this hypothesis. In contrast, the literature consistently demonstrates the value 
of degrees, particularly bachelor’s degrees. For these reasons, it might be argued that 
even when students do not seek degrees, community colleges should seek to raise their 
degree aspirations, and help them to recognize the opportunities for advancement in 
education and subsequently in employment with further education. 

The disappointing completion data for low-income and minority students presents a final 
challenge to the argument that completion at community colleges should not be 
emphasized. Even after controlling for high school test scores, other personal 
characteristics, and stated degree goals, socioeconomic status continues to be strongly 
related to the probability of completion. If this fact represents systematic difficulties 
faced by lower income students, then colleges should try to do something about those 
difficulties. Alternatively, if it represents systematic differences in aspirations, even after 
controlling for high school academic record, then we should ask why low-income 
students have lower aspirations. 

Research on the Effects of College Programs on Persistence and Completion 

There is a tremendous amount of research on persistence and completion in higher 
education but few concrete insights about the specific effects of institutional policies on 
community college retention and completion. Moreover, two fundamental problems with 
the research literature in this area compromise the usefulness of research findings; one is 
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theoretical or conceptual and the other empirical. Still, some useful approaches to 
increasing retention and completion have been identified. 

The conceptual problems generally result from attempts to apply, to community colleges, 
models of student retention and completion developed in the study of four-year 
institutions, whose student subjects are usually of traditional age, residential, and in BA- 
granting programs. Community college students, conversely, are likely to be older, 
working, and attending part time. The implication of the typical models is that 
community colleges should work towards making the community college experience as 
similar to the residential four-year experience as possible, but, in fact, a very different 
approach may be more effective for typical community college students. 

Little high-quality national data are available for analyzing institutional practices: most 
provide little detail on specific institutional practices; do not provide information on a 
single college, which would permit evaluation of its characteristics, policies, and 
practices, and their effect on student success; and are of mixed quality. 

The methodological problems in the research have several sources. The studies on 
persistence vary significantly in terms of the definitions given to persistence and 
completion. In addition, the current literature employs a wide range of datasets, including 
single-institution data, system-wide (state-level) data, and nationally-representative 
datasets, and the type used has implications for the interpretation of results. The large 
majority of research on effective practices in community colleges appears in unpublished 
reports that are both not widely disseminated or reviewed formally for quality. 

Perhaps the most serious and difficult-to-solve problem concerns the attribution of 
causality in the evaluation of programs. Most practices that are studied are discrete 
programs in which some, but not all, students participate. Studies of the effectiveness of 
the programs generally consist of a comparison between participants and non- 
participants, although students were not randomly selected for each group and therefore 
may have personal characteristics that impact on the effect of the program on them. 
Experimental studies that assign students at random to program and comparison groups 
can address many of these problems, but these are rare, difficult to implement, and 
extremely expensive. 

Nevertheless, considerable progress can be made with the resources and data now 
available. Multivariate analysis controlling for measurable student characteristics is a 
first step. In addition, reports must provide descriptions of the programs themselves as 
well as the process through which students are recruited and enrolled in the programs. 
This information allows the readers to evaluate the extent to which program and 
comparison students might differ. But most studies of community college practices fail to 
provide this type of information. 

Given the wide variation in the quality of research on effective practices, what 
conclusions can be drawn from the existing literature? 
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Review of advising, counseling, mentoring, and orientation programs in several studies 
demonstrate mixed results for individual programs. Organizing these services into a 
single support center, however, did increase the measured effectiveness of support 
services. In addition, a review of the federally sponsored Student Support Services (SSS) 
programs showed that peer tutoring, workshops, and cultural events are effective program 
components, with peer tutoring shown as particularly effective. 

Learning communities, which organize instruction around themes and allow students to 
go through a program as cohorts, constitute a particularly interesting model for 
community colleges since it is one way that these commuter institutions can engage with 
their students in a more intensive way than normally occurs in classrooms. There is 
extensive research on learning communities, although much of that research is focused on 
four-year colleges. Since learning communities usually involve coordination of several 
classes, the typical community college working and part-time students may have 
difficulty participating. (Four year college students are much more likely to be attending 
full time or even to be in residence.) And the extensive reliance on part-time faculty at 
many colleges may also thwart the close coordination required of the faculty who 
participate in the program. Nevertheless, research on community college programs has 
generally been favorable. Preliminary results from a random-assignment study show 
positive results. 

The provision of developmental education services for academically under-prepared 
students is a fundamental function of every community college. In many colleges, the 
majority of students arrive with academic skills that are judged to be inadequate for 
success in college-level courses. Developmental education consists of some or all of the 
following activities: instruction in academic skills, advising, counseling, and 
comprehensive support services. In some colleges, research suggests that students who 
start in developmental courses or programs persist and graduate at rates similar to those 
experienced by students who started in regular college-level courses. But research on the 
effectiveness of developmental education suffers from serious methodological problems. 
Given the importance of developmental education to the mission of community colleges, 
this is an area that needs more attention. 

Finally, much of the research on effective practices is concerned with discrete 
“programs” often involving only a limited number of students. Even if such programs 
are successful, they may be difficult to implement on a college wide basis. This suggests 
that attention to college-wide reform is necessary, yet there is little research that 
evaluates broad institutional reform or that identifies the characteristics of college that 
have been relatively successful in improving retention and graduation. There are 
encouraging examples of colleges in which some measures of student success have 
improved after the enactment of whole-school reforms involving changes in organization, 
teaching methods, counseling and student services, relationships to the community, and 
organizational philosophy. But given the importance of this issue, it deserves much more 
systematic attention. 
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Institutional Characteristics Associated with Completion and Persistence 

This report is a first step in a longer process designed to identify institutional 
characteristics and policies that are related to improved graduation and persistence. There 
are, of course, many factors that might cause variation in student success and many of 
them may be beyond the control of an individual institution. Some such factors relate to 
the personal characteristics of their students, such as part-time attendance, which has 
been shown to hinder completion. These factors must also be taken into account when 
graduation rates are used to judge the performance of an individual college, because 
failing to control for students’ academic readiness unfairly penalizes colleges that enroll 
less well-prepared students and gives undeserved credit to those with selective 
admissions policies. 

Research Findings 

Literature on institutional characteristics that promote student success consists mainly of 
studies of four-year institutions, so some findings are not directly applicable to 
community colleges and their students. Nevertheless, the studies identify these school 
factors as positively related to student persistence: students with higher SAT scores, 
students with a higher family income, a higher percentage of female students, the 
availability of student housing and a large percentage of students living in it, a smaller 
student body, and greater expenditures on instructional and academic support. Factors 
negatively affecting college completion have been identified as student bodies consisting 
of a large number who attend part time, are members of minority groups, and are older. 

Studies, therefore, consistently find that the typical characteristics of community college 
students are also those characteristics that predict lower graduation rates. Since students 
with such characteristics are precisely the students that community colleges often enroll 
under their open access policies, any attempt to improve graduation rates by becoming 
more selective, while possible, would violate this underlying mission of the colleges. 

To investigate whether the findings from these research studies apply to community 
colleges, we conducted our own analysis of institutional graduation rates at community 
colleges, using data from the National Center for Education Statistics’ Graduation Rate 
Survey (GRS) — often referred to as the Student Right-to-Know (SRK) data, which is part 
of IPEDS. To compute this graduation rate, colleges report the percent of a cohort of 
first-time, full-time (FTFT) students in degree programs that earns an associates degree 
or certificate within three years of initial enrollment. We also computed a “predicted” 
graduation rate based on this analysis. This is the graduation rate that we would expect a 
college to have, given its institutional characteristics. We then compared actual and 
predicted rates. When the actual rate was higher than the predicted rate, we concluded 
that the college is, in effect, performing at a higher rate than we would expect. 

This methodology has been increasingly used to analyze the performance of four-year 
colleges. It is now used as part of the U.S. News and World Report four-year college 
rankings. But this is the first time that it has been used to analyze community colleges. 
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We found that, overall, 22.3 percent of FTFT degree-seeking community college students 
in the sample attained a postsecondary credential in their starting institutions within three 
years. Graduation rates for women were higher than for men; and African-Americans, 
Hispanics, and Native Americans all graduated at lower rates than Whites. Contrary to 
findings at four-year institutions, Asian students in community colleges were also less 
likely to earn a degree or certificate than were Whites. 

To analyze the potential different influences on the various community college 
graduation rates we created three models, each with “explanatory variables” that 
considered different college and student characteristics including urbanicity; enrollment 
size; the percent of enrollments accounted for by African-Americans, Native-Americans, 
Hispanics, part-time students, and women; federal aid per student; and several financial 
variables. We found that large colleges had lower graduation rates. Colleges with larger 
shares of African-American, Hispanic, and part-time students, and those with higher in- 
state tuition and lower instructional expenditures all had lower graduation rates. Colleges 
that conferred relatively more certificates had higher graduation rates because certificates 
are shorter and easier to complete than associates degrees. We found some unexpected 
results. Colleges with higher proportions of women and Asian students had lower 
graduation rates, and higher per student expenditures on student services were also 
associated with lower graduation rates. It may be that colleges with students facing more 
serious problems tend to spend more on student services so that any positive effect of the 
services may be offset by the more problematic characteristics of the students. 

In addition to identifying institutional characteristics that are associated with higher 
completion rates, this methodology can also be used to identify colleges and states that 
perform at higher levels than expected (after measurable characteristics are taken into 
account). The analysis therefore can be used to benchmark both state-level and college- 
level performance. We illustrate this approach for states and colleges participating in the 
Lumina Achieving the Dream initiative. 

There are many factors that are not measured or included in available data sets. However, 
identifying colleges or states that over- or under-perform, with respect to their measured 
characteristics, can be the basis of more qualitative case studies of states and institutions 
designed to identify additional institutional and policy factors that promote graduation 
and retention. 

Therefore, this approach can make a promising contribution to understanding and 
improving college performance. Future research will refine the graduation rate measures 
and evaluate their usefulness in comparison to much more detailed and comprehensive 
measures of performance available in some states. 

Conclusions and Recommendations 

This report shows that low-income, minority, and other underserved students are over- 
represented at community colleges, and that they complete degrees and certificates at 
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relatively low rates. We have discussed community college retention and completion 
from various perspectives, including the argument about whether graduation rates are an 
appropriate index forjudging the colleges. We have also reviewed institutional practices 
designed to increase college completion — such as enrollment size and student 
demographic and financial characteristics — and used them to explain community college 
graduation rates. Finally, we have used this analysis to develop an index that can be used 
to benchmark college graduation rates. 

On the basis of our findings, we offer a set of programmatic and policy implications: 
Retention, Graduation, and Transfer Rates 

(1) Colleges must recognize the need to improve retention, graduation, and transfer 
rates. We acknowledge the validity of many of the points about why more 
students do not graduate or transfer, but having good explanations for why some 
students never finish a degree or certificate is not the same as saying that 
graduation rates should not be higher. And, most colleges are indeed trying to 
increase retention. In fact, some colleges have been relatively successful at 
overcoming many of the barriers that all colleges face. 

Practice and Policy 

Despite a large amount of research on retention in higher education, we have much to 
learn about the effects of institutional practices on retention and completion at 
community colleges. While there are real weaknesses in the research, and limitations in 
the application of research findings on four-year institutions for community colleges, 
there are still lessons to be learned. 

(2) Current research does provide support for the effectiveness of learning 
communities. Learning communities have attracted a great deal of attention in the 
last decade and the research on their effectiveness is generally positive. They may 
be particularly significant for community colleges, and for students in 
developmental education, since the classroom may offer the only opportunity to 
engage the students with the institution. At the same time, though, it is important 
for colleges to work out ways to organize learning communities to enable the 
participation of a wide range of students, including part-time students and those 
who work. 

(3) Research on counseling, advising, and student orientation suggests that all can be 
effective for retaining students, but many questions about design and intensity 
remain. Orientation and first-year seminars appear to be effective, but much of the 
relevant research concerns four-year colleges. Many practitioners are convinced 
that comprehensive services that include personal and career counseling and 
academic advising are effective. Yet, answers to these questions are needed: what 
is the role of faculty, what should constitute the educational preparation of 
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counseling and advising staff, how can services be provided without excessive 
cost, and how much counseling can be done on-line? 

(4) Tuition levels, instructional expenditures, and institution size are related to 
graduation rates. Our research suggests that higher tuition, lower instructional 
expenditures, and larger enrollments are related to lower graduation rates. It is 
unrealistic to propose cutting the size of colleges, but the result here does suggest 
that it would be interesting to try to explain what smaller colleges do that might 
influence retention. Indeed, an extensive discussion of the effects of school and 
classroom size at the K-12 level has been occurring for some time. 

(5) It is essential to promote a more thorough discussion of the determinants of 
student outcomes and the effects of programs and policies on those outcomes. 
There has been a tremendous amount of discussion of effectiveness and student 
outcomes, and some reports contain descriptive numbers. Accountability-related 
reports often also contain enrollment and graduation percentages for particular 
demographic groups. Still, the published reports of college practices rarely 
contain thorough discussions or analyses of outcomes that address questions of 
causality. In addition, colleges and community college organizations need to 
provide more opportunities for faculty and administrators to exploit the richness 
of the quantitative and institutional data that are available to them. 

Research 

The core strategy of the Achieving the Dream initiative is the use of data and analysis to 
identify and remove barriers to increasing student success at community colleges. There 
is a widespread consensus within the initiative and, indeed, throughout the leadership of 
the community college movement in the United States that colleges must be more “data 
driven,” and shift from a “culture of anecdote” to a “culture of evidence.” There is less 
consensus about exactly what constitutes “evidence.” While many programs are 
identified as “best practices” and, in some cases, there appears to be a consensus about 
“what works,” this report has argued that a rigorous look at the underlying research yields 
less than definitive conclusions. 

Research on community colleges faces two important barriers. First, the large majority of 
research on higher education concerns four-year colleges. Second, assessing the 
effectiveness of educational initiatives is thwarted by difficult methodological problems. 
Many community colleges do not have the resources to do evidence-based research, and 
some administrators may not be convinced that it is necessary. 

(6) It is necessary to recognize that assessments of the effectiveness of practices are 
difficult and involve a continuum of activities and analyses that range from simple 
descriptive comparisons to more time-consuming and expensive controlled 
analyses and experiments. Descriptive comparisons of outcome measures for 
participants and non-participants are a useful beginning to any analysis, but more 
definitive research requires careful attention to the characteristics of participants 
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and non-participants, and to the process through which students enroll or are 
recruited into relevant programs. 

(7) Studies must pay increased attention to college-wide changes and the 
institutionalization of promising practices. Research focused on practices to 
improve retention and completion tends to analyze specific programs. Other 
innovations, such as organizational changes or new hiring and professional 
development policies, are more difficult to isolate for assessment purposes. 
Moreover, even if discrete practices are successful for a small number of students, 
they may not be easily “brought to scale” or institutionalized. Indeed, a successful 
institution-wide reform may not look anything like a discrete program. 
Researchers, college administrators, and faculty need to focus more attention on 
college-wide innovations and develop methods to study them and identify 
successful strategies. 

(8) Community colleges must recognize the resource needs of institutional research. 
While it is easy to agree with the need to strengthen a culture of evidence in 
principle, it may not seem so obvious if a college president is faced with a choice 
between strengthening the developmental education services and increasing the 
institutional research function. Institutional research must be seen as an 
investment that will make other resources more effective. 

(9) More systematic methods to publicize and disseminate useful research findings 
from state and institutional research offices must be developed. State and college- 
level researchers frequently carry out analyses and assessments of policies and 
practices, but often they are not published in ways that make them useful to wider 
audiences. Many reports of “best practices” fail to provide enough backup 
information and data for readers to make a judgment about the relative 
effectiveness of the practice. Assessment results are often unavailable to 
researchers. The type of discussion that would arise from more systematic 
dissemination of local and state research results should be seen as a central 
element of an increased emphasis on a “culture of evidence.” 

(10) Researchers must develop models designed specifically to study community 
colleges. The student integration model, developed to analyze student persistence 
at four-year institutions, has been the most influential perspective on student 
retention, but it has only limited applicability for commuter schools and, 
particularly, for community colleges. Researchers and college faculty and 
administrators need to consider alternative approaches and frameworks. 

(11) The wide variation in college performance must be exploited to develop insights 
about effective strategies and policies. We have shown that there is a wide 
variation among community colleges in measured cohort graduation rates, and 
have developed a benchmarking index that can be used to compare graduation 
rates. As expected, for the most part, these measures are also related to college 
size, demographics, and financial characteristics. Nevertheless, variations in 
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college performance offer an as yet unexploited opportunity to examine, using a 
variety of methodologies, the effectiveness of different institutional strategies and 
characteristics. 

(12) Collaboration between academic, institutional, and state-level researchers should 
be promoted. Researchers working to improve the performance of community 
colleges face formidable problems. They will have more chance of success if they 
use a variety of methodologies and if they combine research based on national, 
state, and local datasets as well as specific institutional and state-level knowledge. 

(13) It is crucial to act now, but question and measure. We have been critical of the 
quantity and quality of research on the effects of institutional practices on 
community college graduation and completion rates, although we have found 
high-quality research on community college practices and sophisticated 
institutional research departments. Nevertheless, relevant results of much of the 
research that we have reviewed are either subject to alternative interpretations or 
published in such a way that makes it difficult to evaluate their validity. 

Therefore, its value as a guide for practice and policy must be carefully 
considered. 

College personnel now have a variety of sources of information — including 
experience, institutional knowledge, and research of varying degrees of 
reliability — and this type of information serves as the basis for action. Colleges 
are, of course, ongoing operations and cannot wait for definitive evidence before 
acting; and, in any case, research should not be viewed in absolute terms — either 
definitive or useless. Still, faculty and administrators should move forward with 
the understanding that questions remain about the effectiveness of what they are 
doing. 

Our suggestion is that, in planning activities, colleges should search for the best 
information they can find, but they should search critically, recognizing that all research 
is not the same and that even the most definitive studies, such as those using random 
assignment methodologies, have limitations. At the same time, they should do what they 
can to monitor progress, and do so in a way as thoroughly and rigorously as possible. The 
interaction between research and practice is not a search for the definitive answer of 
“what works.” Rather, it is a constant and continuous process and conversation within 
and among the colleges, and with outside researchers and policymakers, as practitioners 
try to improve their practice in the context of a constantly changing environment. We 
urge that, in this conversation, colleges use the best possible data and the most 
appropriate methodologies. The process of continuous discussion is probably better 
described as a “culture of inquiry” rather than a “culture of evidence.” When speaking of 
monitoring disarmament treaties, Ronald Reagan said, “Trust, but verify.” When 
developing and implementing innovations to improve student outcomes, we say “Act 
now, but question and measure.” 



12 




IMPROVING STUDENT ATTAINMENT IN COMMUNITY COLLEGES: 
INSTITUTIONAL CHARACTERISTICS AND POLICIES 



I. Introduction 

Community colleges are a crucial point of access for low-income and minority students, 
who are overrepresented (with respect to their share of undergraduate enrollment) in two- 
year and less-than-two-year postsecondary institutions (Cohen & Brawer, 2003; Rouse, 
1995). The concentration of low-income and first-generation college students in 
community colleges is particularly striking. In the 1995-96 school year, for example, 
students from the lowest half of the family income distribution accounted for 55 percent 
of all first-time college students enrolled in two-year public institutions, but only 35 
percent in four-year public and not- for profit institutions. 1 The community college access 
mission is built on low tuition, convenient location, flexible scheduling, an open-door 
admissions policy, and programs and services designed to support at-risk students with a 
variety of social and academic barriers to postsecondary success. Therefore, if 
community colleges — or similar institutions — were not available, many of these students 
would not be in college (Alfonso, 2004; Rouse, 1995). 

While community colleges have played a central role in opening access to college to a 
wide variety of students, access alone is insufficient. Students must also be successful 
after they have enrolled. Consequently, in the last decade, policymakers, educators, 
accreditors, and scholars have increasingly turned their attention to persistence and 
completion among community college students. Graduation rates have long been a 
central preoccupation of educators, yet retention and completion at community colleges 
specifically have attracted much less attention even though many community colleges 
students never finish a degree. Only 36 percent of students who enrolled in a community 
college as their first postsecondary enrollment in the 1995-96 school year had completed 
a certificate, associate, or bachelor’s degree within six years. Another 22 percent were 
still enrolled in college (about three-fifths of whom were enrolled in a four-year 
institution). 2 Therefore, 42 percent of students who started in a two-year public institution 
had left college within six years after initial enrollment without a degree or certificate. 
Low-income, minority, and first-generation college students all have even lower six-year 
completion rates. Those among these populations who do complete tend to earn lower 
level credentials — for example, certificates rather than associate and bachelor’s degrees. 

This report is part of a larger initiative funded by the Lumina Foundation for Education. 
The initiative, Achieving the Dream: Community Colleges Count, will initially work with 
27 community colleges in five states to help them increase retention, completion, and 
success for low-income students, students of color, first-generation college students, and 
other underserved groups. 3 



'Authors’ calculations from the Beginning Postsecondary Student Longitudinal Study of 1995-96. 
2 Authors’ calculations from the Beginning Postsecondary Student Longitudinal Study of 1995-96. 
3 For more information on Achieving the Dream, see 
http://www.luminafoundation.org/newsroom/news_releases/060104.html. 
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The report has four main goals, each addressed in a separate section. The first goal, 
addressed in the next section, is to provide both general descriptive information on 
community college student characteristics and educational outcomes and make 
comparisons with other higher education sectors. Thus, we review data on community 
college enrollment and completion, establishing that community colleges provide access 
to a broad range of students, but that many of them leave without degrees or certificates. 

In the following section, we discuss the importance of college completion and transfer, 
reviewing the controversy about whether completion is a meaningful or justifiable 
standard for a community college. Community college faculty and administrators often 
argue that students attend their institutions for many reasons other than completing 
degrees or certificates, so evaluating colleges or judging student success on the basis of 
degree completion is misleading and unfair. 

Our third goal is to review the state of research on the determinants of student outcomes 
in community colleges. This review provides some programmatic guidance for colleges 
working towards improving their student outcomes. Just as important, this section of the 
report reveals that current research provides surprisingly few definitive conclusions about 
the effect of institutional programs and practices on community college educational 
outcomes. While there is extensive literature on completion at four-year colleges, there 
has been much less analysis of community colleges. Moreover, the existing research on 
community colleges has important theoretical and empirical weaknesses. Thus, this 
section provides some programmatic guidance and also forms the basis for suggestions to 
improve research on student attainment in community colleges. 

Our fourth goal is to initiate our own program of empirical research on institutional 
graduation rates. We begin this effort by conducting an analysis of community college 
graduation rates. As part of it, we develop a benchmarking system for evaluating college 
performance. The system is based on college graduation rates and takes account of 
differences in college characteristics. Therefore, it is a more meaningful and fairer 
measure than raw graduation rates and it forms the basis of subsequent research using 
both quantitative and qualitative methods to identify institutional practices that are related 
to improved student outcomes. 
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II. Community College Enrollment and Degree Completion 

In this section, we present an overview of postsecondary enrollment and degree 
completion trends, placing particular emphasis on minority and low-income students. 
Using data from the Integrated Postsecondary Education Data System (IPEDS), we 
describe the higher education system in terms of enrollment and completion rates at 
different types of postsecondary institutions. IPEDS data are collected annually by the 
National Center for Education Statistics (NCES) from all postsecondary institutions in 
the country. Since the data provide only a snapshot of enrollments and completions at a 
point in time, 4 they do not supply information on the demographic characteristics, 
experiences, and outcomes of individual students over time. Moreover, IPEDS does not 
include data on student family income nor other socioeconomic characteristics, such as 
parental education. Thus, in order to examine the persistence and graduation experience 
of minority and low-income students, we relied on data from the Beginning 
Postsecondary Students Longitudinal Study of 1995-96 (BPS96). This survey is based on 
a sample of students who started college for the first time in the 1995-96 school year. The 
students in the sample were re-interviewed in 1998 and 2001, providing information on 
what happened to them six years after their initial enrollment. 5 

Community College Enrollment 

Data from IPEDS indicate that in the decade following 1 992 undergraduate enrollment 
grew by 1 1 percent, totaling almost 15 million students by the fall semester of 2002 
(Figure 1). All of that increase was accounted for by minority students, since college 
enrollment for White students actually fell by more than 600,000 students. The growth of 
Hispanic enrollment was particularly strong, increasing more than 50 percent to over 1.7 
million students. Some of this growth can be attributed to the overall growth of the 
Hispanic population in the United States during the 1990s. 



4 As part of the IPEDS protocol, colleges provide data on the percentage of a cohort of first-time, full-time 
community college students starting in a particular year who graduate within three years. We will discuss 
these data later in this report. 

5 Note that IPEDS sample includes all students, while the BPS96 sample includes only first-time college 
students. Although many first-time students are non-traditional in that they may have delayed 
postsecondary enrollment after high school, the BPS96 sample will contain proportionately more traditional 
(non-delaying) students than the IPEDS sample. 
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Figure 1. Undergraduate Fall Enrollment by Race/Ethnicity: 1992, 1997, and 
2002 




Source: Authors’ calculations using data from IPEDS, Fall Enrollment Surveys: 1992, 1997, and 2002. 



Neither the overall number of students nor the demographic subgroups are evenly 
distributed among institutions. Table 1 displays the share of enrollments by race among 
the nine categories of postsecondary institutions enumerated by NCES. NCES 
distinguishes among three types of institutional control — public (pub), private not-for- 
profit (p-nfp), and private for-profit (p-fp). NCES also distinguishes among three degree 
levels. Colleges that confer the bachelor’s degree are categorized as four-year institutions 
(4yr). Institutions whose highest degree is an associate degree are considered two-year 
institutions (2yr), and those whose highest degree is lower than an associate degree are 
labeled less than two-year institutions (<2yr). 6 In all of these cases, the categories are 
based on the highest degree conferred, so a four-year institution can also confer associate 
and lower degrees. 



6 We acknowledge that these labels are misleading. For many institutions, most students take longer to 
complete their degrees than these labels suggest. Nevertheless, we use these terms in order to maintain 
consistency with the NCES usage. 
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Table 1. 



Fall 2002 Undergraduate Enrollment by Institution Type and 
Race/Ethnicity, Column Percents 





All 


African 

American 


Hispanic 


Asian 


White 


Other 


4yr pub 


35 


31 


25 


37 


38 


30 


4yr p-nfp 


16 


14 


13 


12 


17 


20 


4yr p-fp 


2 


3 


2 


2 


1 


8 


2yr pub 


42 


44 


51 


46 


41 


38 


2yr p-nfp 


- 


- 


1 


- 


- 


1 


2yr p-fp 


2 


3 


2 


1 


1 


2 


<2yr pub 


1 


1 


1 


1 


1 


1 


<2yr p-nfp 


- 


- 


- 


- 


- 


- 


<2yr p-fp 


2 


3 


5 


1 


1 


2 



Source: Authors’ calculations using data from IPEDS Fall Enrollment Survey 2002. 

Notes: Totals do not sum to 100% due to rounding. 

Dash (-) rounds to 0 percent. 

Two-year public institutions (henceforth called community colleges) account for 42 
percent of total enrollments; four-year institutions (public and not-for-profit together) 
account for 51 percent of total enrollments (Table l). 7 Thus, all other types of institutions 
together enroll only seven percent of all college students. Note that community colleges 
are particularly important for Hispanics, African-Americans, and Asians. These three 
minority groups are all overrepresented in two-year institutions (i.e., 5 1 percent of all 
Hispanic college students are enrolled in community colleges compared to only 42 
percent of all postsecondary students). 

Enrollments by Socioeconomic Characteristics 

Figures 2 and 3 display the enrollment distribution among the nine NCES college 
categories by family income and parental education for all first-time postsecondary 
students in 1995-96. These measures of socioeconomic status are strongly related to the 
patterns of enrollment. Lower income students and those whose parents have less 
education are underrepresented in four-year institutions and overrepresented in two-year 
and less-than-two-year colleges. Community colleges enroll the largest number of low- 
income and first-generation college students (those whose parents have a high school 
diploma or less), but these students account for a larger share of enrollments at for-profit 
institutions than at community colleges. 



7 

Note that these data are based on a student headcount at one point in the fall of 2002. If all students who 
enroll for credit in a postsecondary institution over the course of a year are counted, community colleges 
alone account for more than 50 percent. Since community college students are much more likely than 
students in four-year institutions to enroll part-time and to interrupt their enrollment over the course of a 
year, fall headcounts fail to count proportionately more community college students. 
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Figure 2. 



1995-96 First-Time Students: Percent in Each Institution Type by 
Household Income Quartile 




■ <2yrp-fp 

□ <2yr p-nfp 

□ <2yr pub 

□ 2yr p-fp 

■ 2yr p-nfp 

□ 2yr pub 

□ 4yr p-fp 

■ 4yr p-nfp 

□ 4yr pub 



Source: Authors’ calculations using data from BPS96. 



Figure 3. 1995-96 First-Time Students: Percent in Each Institution Type by 

Parents’ Level of Education 



■ <2yr p-fp 

□ <2yr p-nfp 

□ <2yr pub 

□ 2yr p-fp 

■ 2yr p-nfp 

□ 2yr pub 

□ 4yr p-fp 

■ 4yr p-nfp 

□ 4yr pub 



H.S. or less Some College BA or higher All 




Source: Authors’ calculations using data from BPS96. 
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In addition to enrollment variations between institution types by socio-economic status 
and race/ethnicity, there are similar distinctions by initial program enrollment within 
community colleges themselves. For example, African-American students are 
particularly concentrated in certificate programs, while Hispanics’ patterns of program 
enrollment at community colleges resemble those of Whites (Table 2). Similarly, low- 
income and first-generation community college students are more likely to enroll in a 
certificate program than in an associate program. Specifically, while 23 percent of 
students in the lowest household income quartile enroll in certificate programs in 
community colleges, only 6 percent do from the highest income quartile. And among 
first-generation students in community colleges, 20 percent enroll in certificate programs, 
while only 5 percent of students whose parents have a bachelor’s or higher degree choose 
a certificate program. 



Table 2. 1995-96 First-Time Students in Community Colleges: Percent in 

Race/Ethnic Category by Initial Degree Program 



Program 


Certificate 


Associate 


White 


14.6 


85.4 


African-American 


26.8 


73.2 


Hispanic 


13.9 


86.1 


Asian/Pacific Islander 


4.4 


95.6 


Native American 


6.5 


93.5 


All 


15.4 


84.6 



Source: Authors’ calculations using data from BPS96. 



Degree Completion and Transfer 

While enrollment trends and patterns by student type are important, since degrees are for 
most students the primary objective of enrollment in higher education, it is very 
important to observe outcome patterns over time and by various student types. Table 3 
uses IPEDS data to present trends in certificate and degree completion in the past decade. 
There are several important findings. First, the number of certificates and associate 
degrees awarded in 2002-03 is almost identical. This is due to a small, yet steady rise in 
associate awards over the decade with a concomitant sharp decline in certificate awards. 
Observe that Whites are primarily responsible for the decline in certificate awards, while 
they are the only race/ethnic category that did not contribute to the increase in associate 
degree awards over the period of observation. In fact, commensurate with the observed 
decline in enrollment among Whites in postsecondary education (see Figure 1), they 
exhibit declines or no change in awards earned from 92-93 to 02-03. Finally, note that the 
number of associate degrees awarded increased slightly faster than bachelor’s degrees, a 
pattern observable for all of the identifiable race/ethnic categories. 
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Table 3. Undergraduate Degrees Awarded by Race/Ethnicity: 1992-93, 1997- 
98, and 2002-03, All Majors 







92-93 




97-98 




02-03 




% change 






Students 


% 


Students 


% 


Students 


% 


92-93 - 02-03 




All 


822,052 


100% 


656,079 


100% 


597,576 


100% 


-27% 




At- Am 


100,839 


14% 


91,520 


14% 


100,325 


17% 


-1% 




Hisp 


83,403 


11% 


83,004 


13% 


103,783 


17% 


24% 


vt UllvdlvS 


Asian 


24,423 


3% 


27,877 


4% 


26,283 


4% 


8% 




White 


408,183 


56% 


348,957 


53% 


321,657 


54% 


-21% 




Other 


116,523 


16% 


104,721 


16% 


45,528 


8% 


-61% 




All 


539,361 


100 


580,591 


100 


604,764 


100 


12% 




Af-Am 


42,956 


8 


56,700 


10 


65,231 


11 


52% 


Associate 


Hisp 


35,862 


7 


49,302 


8 


63,409 


10 


77% 


degrees 


Asian 


16,581 


3 


25,358 


4 


30,492 


5 


84% 




White 


404,209 


76 


417,025 


72 


403,339 


67 


0% 




Other 


34,491 


6 


32,206 


6 


42,293 


7 


23% 




All 


1,189,001 


100 


1,203,827 


100 


1,313,614 


100 


10% 




Af-Am 


77,357 


7 


96,091 


8 


111,686 


9 


44% 


Bachelor’s 


Hisp 


58,229 


5 


78,270 


7 


95,681 


7 


64% 


degrees 


Asian 


50,891 


4 


70,167 


6 


79,634 


6 


56% 




White 


937,545 


79 


880,351 


73 


917,739 


70 


-2% 




Other 


62,058 


5 


78,948 


7 


108,874 


8 


75% 



Source: Authors’ calculations using data from IPEDS Degree Completion Surveys 1992, 1997 and 2002. 

Note: Total number of observations do not sum to 100% due to degree completers for whom 

race/ethnicity was not indicated. 



Using the BPS96 sample, Figure 4 displays the highest educational outcome, six years 
after first enrollment, for students by initial institution type. After six years, 36 percent of 
community college students had completed either a certificate, associate, or bachelor’s 
degree. An additional 13 percent had transferred to a four-year institution (without yet 
earning any degree). Degree completion rates are much higher among students starting in 
four-year institutions and, not surprisingly, a much larger share of those degrees are 
accounted for by bachelor’s degrees. Observe also the higher completion rates at private 
not-for-profit and for-profit two-year institutions, relative to that of community colleges. 
The figure also clearly shows the relative emphasis on certificates among both the two- 
year and four-year for-profit institutions. 
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Figure 4. 



1995-96 First-Time Students: Highest Outcome after Six Years by 
First Institution Type 



80% 

70% 

60% 

50% 

40% 

30% 

20 % 

10 % 

0 % 




4yr pub 4yr p-nfp 4yr p-fp 2yr pub 2yr p-nfp 2yr p-fp 



□ Bachelor's 

□ Transfer 
■ Associate 

□ Certificate 



Source: Authors’ calculations using data from BPS96. 



Race/Ethnicity, Household Income and Outcomes of Community College Students 

Within community colleges, minorities are less likely than Whites to complete a degree 
or certificate (data not shown here). Completion rates among African-Americans are 
particularly low. Only 27 percent earned any type of award within six years, while an 
additional 10 percent transferred to a four-year institution. The vast majority of those 
awards are certificates. Hispanics complete at a slightly higher rate (29 percent), and over 
two-thirds of their awards were accounted for by associate and bachelor’s degrees. 

Figures 5 and 6 indicate predictable degree completion patterns by income and parental 
education levels for first- time students at community colleges. However, contrary to 
expectations, students from the second income quartile complete at lower rates than those 
from the lowest quartile, and completion rates for first-generation college students 
actually exceed those for students with at least one parent who had some college. 
Nonetheless, both the latter group and those in the second income quartile had higher 
rates of both transfer and bachelor’s degree attainment. As might be expected, 
community college students from the top half of the family income distribution and those 
with at least one parent with a bachelor’s degree are much more likely to earn some 
award — and these awards tend to be degrees rather than certificates. In addition, high 
socioeconomic status (SES) students who had not earned an award within six years were 
more likely to have transferred to and still be enrolled at a four-year college. 
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Figure 5. 



1995-96 First-Time Students: Highest Outcome After Six Years by 
Household Income Quartile 




Source: Authors’ calculations using data from BPS96. 



□ Bachelor's 

□ Transfer 
■ Associate 

□ Certificate 



Figure 6. 1995-96 First-Time Students: Highest Outcome After Six Years by 

Parents’ Level of Education 



80% 



70% 



60% 

50% 

40% 

30% 

20 % 

10 % 

0% 



10 % 



13% 



17% 



10 % 



All Students 



6% 



8 % 



17% 



12 % 



H.S. or less 



9% 



17% 



13% 



8 % 



19% 



21 % 



20 % 



7% 



Some college B. A. of higher 



□ Bachelor's 

□ Transfer 
■ Associate 

□ Certificate 



Source: Authors’ calculations using data from BPS96. 
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Measuring Factors Affecting Completion and Transfer 

The data that we have presented so far have shown the relationship between 
race/ethnicity and family SES measures on the one hand, and persistence and degree 
completion on the other. The availability of nationally representative longitudinal data 
allows for analysis of completion rates while simultaneously taking into account many 
variables. For example, it may be that minority students graduate at lower rates primarily 
because they tend to come from lower income families. Therefore, it is important to 
review carefully what existing empirical research, using more comprehensive analytical 
techniques, reveals about the effects of individual characteristics on persistence and 
degree completion. 

Findings from such research indicate that African-American and Hispanic community 
college students earn certificates or degrees and transfer at lower rates than do Whites. 
Yet this effect is weakened in studies that control for high school background and family 
income. Community college students with higher family income and whose parents are 
college educated are more likely to complete degrees and transfer than those who are low 
income and the first in their family to attend college. Furthermore, community college 
students with higher secondary school grade point averages and assessment test scores 
are also more likely to earn a degree (Bailey, Alfonso, Scott, & Leinbach, in press). 

One of the most common findings in research on student success in college is that 
attendance patterns have a strong influence on student completion (Horn & Carroll, 

1996). For example, consistent with the findings of earlier institution-level research, 
Adelman (1999) found that delaying enrollment after high school, attending part time, or 
interrupting enrollment significantly decrease the chances that a student will earn a 
bachelor’s degree. 

These relationships are also important for community college students. Those who delay 
entering college after high school, work while attending school part time, and interrupt 
their schooling are less likely to earn degrees (Bailey, Alfonso et al., in press). 
Community college students in general are much more likely to follow these non- 
traditional patterns of attendance than students at four-year colleges, so it is significant 
that the attendance patterns that put students most at risk of failing to persist are those 
that characterize most community college students (Bailey, Leinbach, et al, in press; 

Horn & Carroll, 1996). Since lower income students also are more likely to delay or 
interrupt their enrollment, or to attend part time, these patterns may be one mechanism 
through which a student’s socioeconomic status influences his or her educational 
outcomes. 

Most studies find that community college students who receive financial aid are more 
likely to persist and graduate than are those who do not receive aid. The findings with 
respect to student loans and debt load are mixed. For example, in a study of semester-to- 
semester persistence of community colleges, Cofer and Somers (2000; 2001) found that 
students with low debt loads are less likely to persist than are those with no debt, but 
those with high levels of debt are more likely to persist. The authors argued that these 
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loan impacts could be related to students’ motivation, suggesting that highly motivated 
students assume larger amounts of debt to meet their graduation goals, while those less 
motivated and insecure about their graduation possibilities take lower levels of loans. 
Thus, Cofer and Somers concluded that changes in financial aid policy — from grants to 
loans — facilitate the persistence of highly motivated students while discouraging the 
persistence of the less motivated. 

Table 4 summarizes the characteristics of community college students that studies most 
often find significantly impact the likelihood of student degree completion or transfer. 



Table 4. Association of Student Characteristic with Degree Completion of 
Community College Students 



Student Characteristic 


Effect on Degree Completion 


Race/Ethnicity 


- (for African-Americans 
and Hispanics, though 
effect weakened when 
controlling for SES) 


Higher Socioeconomic Status 


+ (effect weakened when 
controlling for attendance 
patterns) 


Better High School Academic Record 


+ 


Non-traditional Attendance Pattern 




• Delayed enrollment after high school 


- 


• Interrupted enrollment or “stopping out” 

• Part time attendance (working while enrolled) 


- 


Receipt of Financial Aid 


+ 


More Household Responsibilities 


- 



Summary 

The data presented in the preceding tables and figures confirm the important role that 
community colleges play in providing access to postsecondary education for low-income 
and minority students. But they also show the relative difficulty that those students have 
in converting their access into certificates and degrees. In addition, the analysis highlights 
some interesting points. For-profit institutions enroll relatively large proportions of 
African-American and Hispanic students. For-profits particularly emphasize certificates, 
and completion rates at these institutions are higher than they are at community colleges. 8 
African-American students have particularly low completion rates and are unusually 
concentrated in certificate programs. Having a high family income or highly educated 
parents significantly increases students’ chances of enrolling in a four-year college and, 
for those who start in a community college, these factors increase their chances of 



8 For a discussion of why for-profits may have higher completion rates, see Bailey, Badway, and Gumport 
( 2001 ). 
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completing an associate or bachelor’s degree, or at least of transferring to a four-year 
college. 

III. Degree Completion and Transfer as Indicators of Student Success 

A central goal of this report is to explore the determinants of college persistence and 
completion in order to help develop public and institutional policy designed to increase 
degree completion among community college students. Several forces are converging to 
bring about an increasing emphasis on community colleges’ completion rates, and it is 
important to consider whether they are the best indicators of student success. Many 
community college students benefit from the skills they learn in college, even if they 
never earn a degree (Bailey, Kienzl, & Marcotte, in press). This positive finding indicates 
the need to consider how much significance should be placed on persistence and 
graduation. In this section we review the growth in the emphasis on graduation and 
discuss the controversy associated with that growth. We conclude that increasing degree 
completion and transfer rates should be central institutional goals, although other 
measures of student success certainly should be considered. 

The Movement toward Outcomes and Accountability 

Several recent trends in many spheres of American society surrounding education are 
leading the public and lawmakers to take a greater interest in measuring the performance 
of institutions of higher education by their student outcomes. And institutions are 
increasingly being held accountable for the results through ratings systems and funding 
levers at all levels of government. 

The popularity of college rankings such as those produced by U.S. News and World 
Report are a reflection of the increasing interest in measures of the quality of higher 
education. Furthermore, accrediting agencies, long criticized for an overemphasis on 
“inputs” such as the credentials of the faculty and number of books in the library, are now 
beginning to focus more on outcomes. The North Central Association of Colleges and 
Schools and other accreditation agencies, for example, now require colleges seeking 
accreditation to give evidence of learning by all of their students, not just by those who 
complete their programs. 

Policymakers are also demanding greater accountability. According to the Rockefeller 
Institute of Government at the State University of New York at Albany, 44 states issue 
annual “report cards” on their colleges, up from 30 in 2000 (“Linking Spending,” 2003). 
More than half of the states now engage in “performance budgeting,” under which state 
officials, in drafting annual budgets, take into account public colleges’ performance. 
Eighteen states have performance funding schemes in which public colleges gain or lose 
set amounts of money based on how well they meet certain standards. 

In the 2004 debate in Congress over reauthorization of the Higher Education Act, the 
Bush administration articulated its desire to hold higher education to higher standards of 
accountability, just as it had with the public schools through the No Child Left Behind 



25 




Act. Under the Higher Education Amendments of 1998, to be eligible to receive federal 
financial aid, colleges are already required to report graduation rates for cohorts of first- 
time, full-time students in 150 percent of the “traditional” graduation period (three years 
for community colleges and six years for baccalaureate-granting institutions). Beginning 
with the Integrated Postsecondary Education Data System (IPEDS) 2002-03 survey of 
graduation rates — what are referred to collectively as “Student Right-to-Know” (SRK) 
statistics — colleges are now required to report overall student graduation rates, as well as 
those for African-American, Hispanic, and female students. 

The Accountability Debate at Community Colleges 

While the focus of policymakers’ increasing demands for accountability has been 
primarily on four-year colleges, community colleges are unlikely to escape increased 
public scrutiny about their outcomes. As competition for public funding intensifies, and 
as community colleges are forced to compete both with other educational institutions — 
including the more politically powerful universities and K-12 systems — and with other 
non-education groups, such as seniors seeking support for health care and retirement 
(Kane, Orszag, & Gunter, 2003), community colleges will increasingly need to measure 
and document their outcomes and the returns on the public’s investment that they 
produce. 

Community college advocates have, however, resisted the use of graduation rates either 
as an accountability measure or as a normative goal. They have advanced three broad 
arguments for this resistance. First, they argue that many students at community colleges 
are not seeking degrees. In many cases, students enroll with the goal of learning some 
specific skills, perhaps to gain a promotion at their current job. Indeed, in response to the 
question on the Beginning Postsecondary Students Longitudinal Study of 1995-96 
(BPS96) survey asking community college students to report their “reasons for 
enrolling,” about 36 percent of the students in associate degree programs said that they 
either wanted to learn “job skills” or had enrolled for “personal enrichment.” 9 From this 
perspective, the range of non-degree outcomes of students may be an indication that 
community colleges are serving multiple community needs, as they are chartered to do. 
Criticizing colleges for low completion rates would reflect a misunderstanding of the 
mission of community colleges and the goals of their students. 

A second reason why community college advocates resist the use of completion as an 
accountability measure is that many factors which may thwart graduation are beyond the 
control of the colleges. Many community college students face serious barriers to success 
in college, such as family and work responsibilities and deficient academic preparation. 
Indeed, it is precisely many such students, who may not have access to four-year 
colleges, whom community colleges seek to serve. But many of these barriers are not 
under the control of the colleges, nor can be mitigated by the institutions. Therefore, 
advocates argue, community colleges should be neither criticized nor penalized for not 
getting many such students to degree completion. 



9 Authors’ calculations from BPS96. 
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A third reason for resisting the use of graduation rates as a performance measure is that 
college students increasingly may attend several colleges before completing their 
degrees. For example, one out of five students in the National Education Longitudinal 
Study of 1988 (NELS) who earned a bachelor’s degree received it from a four-year 
college other than the one in which they were initially enrolled (Adelman, 2003). And 
findings from BPS96 indicate that up to 40 percent of first-time community college 
students attended more than one institution during their six years of observation in the 
survey. 10 Adelman points out that students change colleges for lots of reasons. In a recent 
article in The Chronicle of Higher Education, he states, “Why should institutions be 
judged for choices, made by students, that are beyond their control? College students are 
legal adults, after all” (Burd, 2004, p. Al). 

These are all important points. Community college students do have varied goals and 
many face particularly challenging barriers that are beyond the control of colleges. 
Finally, many also transfer to four-year or to other two-year colleges without earning a 
degree, reducing a college’s graduation rate — even though these outcomes may be 
positive for students. Thus, institutional graduation rates do underestimate individual 
graduation rates. This distortion is not very large for the three-year period that the NCES 
uses in its Student Right-to-Know data, but institutional and individual graduation rates 
do diverge substantially as the time period increases to six years. 11 

Furthermore, while institutional graduation rates can be taken as underestimates of 
individual graduation rates, there is still useful information to be gleaned in examining 
differences among colleges in institutional graduation rates. Explaining why some 
colleges with similar characteristics and similar types of students have much higher rates 
than others might offer insights into possible policies and practices that could improve 
performance in many colleges. In addition, any individual college can analyze its 
completion and persistence rate as an important outcome of programs aimed at improving 
its performance. 

Although — as already stated — institutional graduation rates taken alone can be 
misleading, there are other factors that suggest that college completion or transfer should 
be important goals for community college students. First, research indicates that earning 
credits without completing a degree does have an economic value, but students get an 
additional financial benefit from the credential (Bailey, Kienzl, & Marcotte, in press; 



10 Authors’ calculations. 

11 We used data from BPS96 to evaluate the difference between institutional graduation rates (the percent 
of an entering cohort that graduates within a given number of years from the institution of first enrollment) 
and individual graduation rates (the percent of entering students who graduate from any institution within a 
given number of years). We found that 23 percent of all first-time, full-time students in degree programs in 
BPS96 graduated (earned a certificate or associate degree) from their institutions of first enrollment within 
three years. This is conceptually equivalent to the Student Right-to-Know rate. Yet 26 percent of these 
students earned a certificate or associate degree at any institution within three years. The comparison 
suggests that the difference between institutional and individual graduation rates is not that large for a 
three-year period. But this difference is much greater for longer periods of time. Over a six-year period, 
while 28 percent of first-time, full-time students in degree programs in the BPS96 sample earned a 
certificate or an associate degree from their initial institution, 46 percent earned a certificate or associate 
degree from any institution. 
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Grubb, 2002). Indeed, for students in academic majors in community college, the real 
benefit comes from eventually earning a bachelor’s degree. There appears to be little 
economic return to transfer-oriented education for students who do not transfer. Earning 
small amounts of credit in academic subjects also has no measured economic value. 
Therefore, research does not refute the argument that short-term course taking to upgrade 
skills can be valuable for students, but neither does it provide strong support for this 
hypothesis. In contrast, the literature consistently demonstrates the value of degrees, 
particularly bachelor’s degrees. For these reasons, it might be argued that even when 
students themselves do not seek degrees, community colleges should strive to raise those 
students’ aspirations, including helping them recognize the opportunities for 
advancement in education and subsequently in employment with further education 
(Jenkins, 2003). 

The student-as-course-taker argument implies that enrollment in an associate degree 
program may overstate students’ goals, since they may only want some specific skills that 
they can learn in a few courses. However, when students are asked about their long-term 
educational expectations, the goals almost always are equal to, and in most cases exceed, 
the level of the programs in which they are enrolled. For example, Figure 7 shows that 86 
percent of students in community college associate degree programs in BPS96 actually 
expected to eventually earn either a bachelor’s (43 percent) or a graduate (43 percent) 
degree. These data suggest that the program in which a student is currently enrolled 
actually understates, rather than overstates, their long-term aspirations. 



Figure 7. 1995-96 First-Time Students in Community Colleges: Highest Degree 

Ever Expected, by Student’s Initial Degree Program 
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Source: Authors’ calculations using data from BPS96. 
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The completion data for low-income and minority students that we presented earlier in 
this report present a final challenge to the argument that degree completion at community 
colleges should not be emphasized. Even after controlling for high school test scores, 
other personal characteristics, and stated degree goals, socioeconomic status continues to 
be strongly related to the probability of completion (Bailey, Alfonso et ah, in press). If 
this fact represents systematic difficulties faced by lower income and minority students, 
then colleges should try to do something about those difficulties. Alternatively, if it 
represents systematic differences in aspirations, even after controlling for high school 
academic record, then we should ask why such students have lower aspirations. 

There is no question that community colleges encounter many challenges that impact 
their ability to increase graduation rates. Nevertheless, completion or transfer should be 
important goals for community college students and thus for the colleges themselves. 
Even among community colleges that face similar challenges, certain institutions perform 
considerably better than others. What do these institutions do to achieve better-than- 
expected results? In the next section, we review the available research to try to answer 
that question. In the subsequent section, we report on our own empirical work to go 
beyond the existing understanding. 

IV. Research Review: 

The Effects of College Programs on Student Persistence and Completion 

What does available research say about institutional practices and policies that can 
increase student persistence and completion at community colleges? There is a 
tremendous amount of research on persistence and completion in higher education. 
Indeed, there are entire journals devoted to this and related topics — for example, the 
Journal of College Student Retention: Research and Practice and the Journal of the 
Freshman Year Experience — and a multitude of unpublished reports from studies on 
individual colleges or state systems. As we shall see, though, this vast landscape of 
papers and reports yields relatively few concrete insights about our specific topic: the 
effects of institutional policies on community college retention and completion. 

Methodological Concerns 

There are two fundamental problems with the research literature in this area. The first is 
theoretical or conceptual and the other empirical. Judgments about the implications of 
existing research need to be made in light of them. We will first discuss these problems at 
a general level and subsequently discuss specific conclusions from the research as they 
relate to the problems. 

The conceptual problems for the most part result from attempts to apply models 
developed in the study of four-year institutions to community colleges. Fundamentally, 
theoretical models of student retention and completion have been developed primarily 
through studies of traditional age, often residential, students in BA-granting programs. As 
we shall see, the logic of these conceptualizations may be much weaker for studying 
typical community college students who are older, more likely to be working, and 
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attending part time. The implication of the typical models is that community colleges 
should work towards making the community college experience as similar to the 
residential four-year experience as possible. It may be that a very different approach is 
more effective for typical community college students. 

Researchers face very serious empirical problems in searching for institutional practices 
that can improve retention and completion. First of all, there are little high-quality 
national data that are useful in analyzing institutional practices. Researchers have two 
options: they can use national data, such as the statistics available from the National 
Center for Education Statistics’ Integrated Postsecondary Education Data System 
(IPEDS), which has little detail on specific institutional practices, or they must rely on 
surveys of students at one college. These individual college surveys have much more 
institutional detail, but they are of mixed quality and, in any case, their results are 
difficult to generalize. Thus we have a much better grasp of the relationship between 
measures of student success and individual characteristics than between student success 
and institutional characteristics, policies, and practices. 

In addition to managing with a lack of good data, researchers must also confront some 
difficult methodological problems. As a result, there is a tremendous range of 
methodological quality within the research literature. Thus, reviewing the research in this 
area is a formidable task. The empirical work on persistence varies significantly in terms 
of the definitions given to persistence and completion, as well as in the datasets used. 
Persistence has been defined as a student’s continuous enrollment towards completion of 
a degree at one particular institution, and it has been measured as within-year persistence 
(semester-to-semester), year-to-year persistence, or persistence to degree completion, 
depending on the duration of the data collection period. Completion refers to degree 
attainment, although sometimes it includes transfers from one college to another (i.e., a 
two-year to a four-year college). 

The current literature employs a wide range of datasets, including single-institution data, 
system-wide (state-level) data, and nationally-representative datasets. The type of dataset 
used has implications for the interpretation of results. For instance, when using data from 
one institution, a student who transfers from a two-year to a four-year college is classified 
as a dropout, but when using national-level data, this student would be classified as 
persisting. 

Rather than analyzing the specific methodological issues from each study, we will 
generally review some of the problems that we have found in our literature search. In 
many cases, samples and variable definitions are not well-defined. Statements are made 
about what successful colleges do without defining how those exemplars were chosen. 
Sometimes reports do not provide much detail about the programs and especially about 
how student participants are selected and recruited. Many studies look at several 
components of an overall program — say counseling and mandatory placement in 
remediation — one at a time and then report the results for each one. But it may be 
possible that those components tend to go together, so it is not clear which one (or both 
together) has the effect, if there is an effect. 
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Perhaps the most serious problem, and also the most difficult to solve, involves the 
attribution of causality in the evaluation of programs. Most practices that are studied are 
discrete programs. Some students are in those programs and others are not. Studies of the 
effectiveness of the programs generally consist of a comparison between those two 
groups of students. But these types of comparisons often do not provide enough 
information to make a judgment. As long as there is some non-random process through 
which students enroll or are recruited to these programs, it may be that any differences 
between participants and non-participants results from the selection process, not the 
effect of the program. Even when authors recognize these problems, many do not take 
available steps that could help to address them. 

There are several steps that researchers can take to purge the analysis of this distortion. 
Multivariate analysis can control for many characteristics. Perhaps the most important is 
some measure of pre-program academic ability. If a program enrolls more successful 
students, then it may not be surprising that program participants are more successful than 
comparison students. Pascarella and Terenzini (1991) generally discuss the strength of 
such controls in the studies that they review, but many studies, especially single- 
institution unpublished studies, do not take this step (or at least they do not report it). 

Even if available variables are included in an analysis, there may be unmeasured 
differences between the two groups. Students who sign up for a program may be more 
motivated than those who do not, even if their measured characteristics are similar, and 
their motivation may be the reason why they sought assistance in the first place. 
Alternatively, counselors may encourage enrollment for students whom they judge might 
“benefit” from a program. 

What can researchers do to address this problem of comparison group differences? 
Fundamentally, the solutions require an analysis or manipulation of the process through 
which students enter a program. One approach is for program organizers to enroll a group 
chosen at random from a pool of applicants, thereby eliminating any systematic 
differences between the two groups of applicants. None of the studies on community 
college practices use this approach, however, although one such study is currently being 
conducted by the research organization, MDRC. A second approach involves conducting 
a statistical analysis of the selection process and using those results to adjust the 
measurement of the program effect (see for example Bettinger & Long, 2004). This is an 
increasingly popular approach in program evaluation, but it requires particular types of 
data that are often unavailable and the results are sometimes difficult to interpret. For the 
most part, this methodological approach has simply not been used in the community 
college literature. 

When neither of these approaches is possible, in addition to controlling for relevant 
measurable characteristics such as academic achievement, researchers must include a 
detailed description of the process through which students enter a program so that at least 
the reader can make a judgment about the potential distortion that the recruitment and 
enrollment process might have on the analysis. If the selection process tends to result in 
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the enrollment of more motivated students, then the measured program effect may be 
exaggerated. Conversely, the enrollment process might result in an understatement of the 
program effect. For example, if counselors consciously send the students with the most 
serious problems to a program, and if students in that program do as well as other 
students with similar measured characteristics, then it may be reasonable to conclude that 
the program was effective. Since many of the studies of programs involve one or a very 
small number of institutions, there is no reason why researchers cannot provide this type 
of information, yet few of the studies reviewed here did so. 12 

Sometimes a reform involves a college-wide change and then presumably any student at 
the college will participate. Evaluations of these types of changes usually involve before 
and after comparisons of measures of student outcomes and they are often informative. 
Nevertheless, it is still important at least to measure the characteristics of incoming 
students and to evaluate whether other changes might have influenced the outcomes. 

There are no easy solutions to these problems. Every approach has practical or 
substantive drawbacks. Nevertheless, given the growing interest in student outcomes, 
considerable progress can be made with the resources and data now available. What is 
most needed is a greater emphasis on thorough and thoughtful analyses that explicitly 
take account of the methodological problems, use available approaches to addressing 
those, and recognize the limitations. 

Conceptual Perspectives on Retention and Completion 

In this section we will discuss in more detail the theoretical approaches to studying 
student retention and how they are used for the analysis of community colleges. Vincent 
Tinto’s (1993) student integration model forms the conceptual basis of much of the 
research on persistence and graduation. It also has some attractive implications for 
institutional policy. Tinto’s 1993 book was written to help colleges understand the 
reasons why students leave so they could design college activities to better serve 
students’ needs and thereby increase retention and graduation rates. Tinto states that 
students’ departures from an institution “reflect the character of the individual’s social 
and intellectual experiences within the institution. Specifically, they mirror the degree to 
which those experiences serve to integrate individuals into the social and intellectual life 
of the institution” (p. 51). The perspective implies that institutions should develop 
processes and activities that will foster greater intellectual and social engagement and 
connections among college students. 

Although the model resonates with educators and portrays an attractive college 
environment, empirical analysis has been difficult. The central concepts of academic and 
social integration are difficult to define and measure. A review of the literature reveals a 



12 See Muraskin (1997) for an example of a study that did try to take account of the selection process. In 
this case, the author provided a detailed comparison of the students who did and did not receive the 
services. This gave some information to allow the reader to judge the reported results. 
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plethora of variables and measures. Several studies use measures of academic and 
social integration that are created by aggregating the effects of multiple variables, but the 
component variables are sometimes not defined and, even when they are, their individual 
effect on student success is hard to disentangle. Overall, there is no consensus about what 
variables to use to measure academic and social integration and how they should be 
combined in empirical analysis. 

Moreover, research on student integration has a profound causality problem. Research on 
four-year colleges shows that students who participate in student organizations or interact 
with faculty graduate at higher rates. Nevertheless, it does not follow that graduation 
rates will rise if every student joins a student organization or interacts with faculty. 
Students may interact with faculty because they share faculty values and an orientation 
towards academic activity. These characteristics could result in a higher probability of 
success even if the student did not have much contact with faculty. Similarly, students 
who choose to be active in a student organization may have certain characteristics that 
would make them more likely to complete a degree with or without that participation. 

Although this problem has not been resolved, the preponderance of research does suggest 
that academic and social integration have positive effects on the persistence of four-year 
college students (Cabrera, Nora, & Castaneda, 1993). In an extensive review of research 
on college student outcomes, Pascarella and Terenzini (1991) found that both the 
frequency and quality of interactions with peers and faculty and the participation in 
extracurricular activities — measures of social and academic integration into the college 
life — positively contribute to students’ persistence at baccalaureate-granting institutions. 
However, even here, the effect is less strong when factors such as student characteristics, 
pre-college experience, and other college experiences are taken into account. 

Tinto (1993) makes clear that, from his perspective, the process of integration goes both 
ways. There is no “single path to enhanced student retention” (p. 201). Thus his 
suggestions with respect to developing student assessment systems emphasize 
understanding the diverse objectives of students. This perspective does not necessarily 
emphasize adherence to one model, but rather to the fit between the student and the 
institution. 

Fit provides a different and broader theoretical framework for understanding students’ 
social and academic integration into campus life. However, research on fit is also plagued 
by methodological problems since the focus has remained on student characteristics and 
not on the relationship between student and institutional characteristics. Measuring 
student characteristics does not indicate how well they fit with what the college has to 
offer, and looking at actual student activities does not necessarily say much about fit 
either. For example, participation in extracurricular activities is usually taken as a sign of 
integration, but the absence of participation in extracurricular activities may reflect a 



13 One study, for example, used 129 different measures for several different concepts — academic 
integration, social integration, external involvement, psychological adjustment, social support, satisfaction 
with the college, positive and negative life events, agreeableness, conscientiousness, and self-esteem 
(Napoli & Wortman, 1998). 
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good fit between a student who is not interested in such activities and a college that does 
not devote resources to providing them. 14 

Much of the research in this area is based on an image of college life typified by the 
residential liberal arts college with multifaceted interactions inside and outside the 
classroom among students and between students and professors. This assumption may be 
problematic for many public, less-selective four-year schools, but it is even less likely to 
be accurate for community college students. Community colleges are commuter 
institutions, a characteristic that makes their students less likely to interact socially 
outside of school hours. In addition, a substantial proportion of community college 
students attend part time. Trying to reproduce the liberal arts, residential ideal may not be 
the best strategy for community colleges, or at least for many students in community 
colleges. Flexible scheduling, convenient transportation, availability of daycare, high- 
quality on-line education, applied pedagogies, and well-designed internships may be 
more important to community college students than the nature of their relationships with 
professors or participation in student organizations. 

These preferences may explain why research on community colleges is much less likely 
to show a positive relationship between measures of integration and student persistence 
and degree completion. Some research has found small positive effects (Bers & Smith, 
1991; Napoli & Wortman, 1998), other studies found no effect, and, in at least one case, 
social integration was found to have a negative effect (Nora, Attinasi, & Matonak, 1990). 
In their review of empirical research on undergraduate student attainment, Pascarella and 
Terenzini (1991) contended that these models do not work as well for commuter colleges 
as residential colleges, stating that “with a few exceptions, the weight of evidence is clear 
that various measures of social integration (including interaction with faculty, interaction 
with peers, and extracurricular involvement) show little if any positive relationship with 
persistence at commuter institutions. This lack of a positive relationship holds regardless 
of the specific measure of social integration used and irrespective of whether or not 
student background characteristics were taken into account in the study design” (p. 402). 

In order to test “fit” at community colleges, empirical researchers need to reformulate 
their conceptions of college life. For example, a student who wants a rich extracurricular 
life will presumably be more likely to persist in a college that offers many such 
opportunities, while a working adult student with many outside responsibilities may 
persist longer in a college that does not devote resources to the typical characteristics of 
“college life.” Alternatively, a highly intellectual student may feel more comfortable in a 
college that emphasizes a traditional pedagogy while a more practical student may thrive 
in a college that emphasizes concrete applications. 

However, research on integration does not measure this alignment, probably because 
good data on student objectives with respect to the types of college experiences that they 



14 Some of the for-profit institutions emphasize convenience over engagement. For example, the University 
of Phoenix has been known for using locations and schedules to facilitate the participation of working 
adults and has not put resources into promoting student involvement through development of 
extracurricular campus activities. 
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want are not available. So, rather than measuring fit, researchers tend to measure student 
characteristics such as GPA or descriptions of student activities such as participation in 
extracurricular clubs and then assume that this is what students want. 

In conclusion, despite methodological concerns, there is a consensus among researchers 
that, on average, measures of academic and social integration and fit positively affect 
persistence and degree attainment at four-year colleges. The conceptual basis for 
applying this model to community colleges is much weaker and, indeed, the findings 
from empirical studies are far less definitive. This is a case where the extent to which the 
research emphasizing four-year colleges has not only reduced the number of analyses of 
community colleges but also distorted the work that has been carried out. Therefore, in 
addition to simply increasing the work on the two-year sector, it is important that 
researchers also develop new conceptual models on which to base that research. 

Since the classroom is the one place where commuter schools do have contact with their 
students, application of Tinto’s model for community colleges has focused increasingly 
on designing classroom environments that promote integration (Tinto, 1997). This 
approach has been realized primarily through an education practice referred to as learning 
communities. These communities will be discussed in the next section. 

Empirical Studies of the Effect of Institutional Management and Practices on 
Student Outcomes 

Much research in the academic literature has been concerned with testing various theories 
of student persistence and completion. Another genre of research asks a more direct set of 
questions. Do students who participate in a particular type of program or activity persist 
or graduate at higher rates? In this section we turn to studies that take this more 
straightforward approach. 

Once again, the large majority of studies of this type are concerned with four-year 
colleges. An analysis of the research on community college practices gives a mixed 
message. There are many studies that do report positive results for various programs or 
policies. These results often agree with the opinions of experienced community college 
administrators and researchers and a general consensus has developed about the value of 
some of those practices. But from the point of view of research, many of the conclusions 
are not well supported or are open to alternative interpretations. To be sure, researchers in 
this area confront difficult challenges resulting from data limitations and methodological 
problems. There are some high-quality studies, but results based on even the best 
methodologies can be open to question. 

While there are many studies on a long list of possible program activities, we will focus 
on three large categories. They include student services, such as advising, counseling, 
mentoring, and orientation programs; learning communities; and developmental 
education. 



35 




Advising, Counseling, Mentoring, and Orientation Programs 

Colleges have been experimenting with various types of advising and student services for 
decades. According to Pascarella and Terenzini’s summary on the effect of such 
programs, “the most consistently effective program format appears to be a first-semester 
freshman seminar that meets as a regular class with an assigned instructor. The purpose 
of this type of seminar is to orient the student to the institution and its programs and to 
teach important academic survival skills” (1991, p. 403). Muraskin and Wilner (2004), in 
their review of institutional practices, also concluded that freshmen year programs were 
effective. Nevertheless, both sets of authors acknowledged that participation in the 
programs was voluntary. Therefore, the positive association might be influenced by 
initial student characteristics and not the services itself. Still the consistency of the 
findings gives more weight to the positive conclusions. 

Evidence on the effectiveness of other types of advising and counseling is more mixed. 
Pascarella and Terenzini (1991) concluded that when students’ pre-college characteristics 
were taken into account, relatively short-term orientation programs had a trivial and 
statistically insignificant direct effect on persistence. They also found mixed results from 
research on the effects on persistence of the amount and quality of academic advising. 
Muraskin and Wilner (2004) reported that persisting students tended to express higher 
satisfaction with counseling services than those who left, although that research on the 
impact of such services was also mixed. They also suggested, though, that measurement 
of counseling effectiveness may be distorted because students who use counseling may 
have come to the college with more problems than those who do not use it. This is a 
situation in which failing to adequately account for differences in the characteristics of 
program participants and non-participants may understate the effectiveness of a program. 

The studies on counseling and advising primarily concern four-year colleges, however. 
Research on these services, and indeed on institutional practices in general, is much 
scarcer for community colleges. For example, in a recent review of attrition research at 
community colleges by Summers (2003), only one out of 14 pages of text discussed the 
effects of institutional programs. The author cited two unpublished single-institution 
studies that generally found positive effects of “matriculation” services such as 
assessment, orientation, and counseling. 

Reports on the Community College of Denver (CCD) have suggested that the college’s 
institutional practices have been effective at increasing retention (Roueche, Ely, & 
Roueche, 2001). CCD’s counseling and academic support services are organized in a 
comprehensive unit called the Academic Support Center (ASC). The college reports that 
the class withdrawal rate was 7.8 percent for students receiving ASC support, while the 
overall campus rate was 12.4 percent. Though this finding is encouraging, the reports do 
not include information that would allow a judgment about the comparability of the ASC 
and other students at the college. 

Student Support Services (SSS), funded under the federal TRIO programs, is perhaps the 
most widespread student services initiative. SSS was evaluated in the mid 1990s 
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(Muraskin, 1997) and the study found that “freshman-year SSS participants increase their 
grade point averages by 0.15 in the first year and 0.1 1 in the second year of college. SSS 
participation also increased retention to the second year of college at the same institution 
by 7 percent and retention to the third year in any institution by 3 percent” (p. 1). 
Interestingly, the study also found that effects increased with increased exposure to SSS 
activities. The author identified peer tutoring, workshops, and cultural events as effective 
components, with peer tutoring shown as particularly effective. This was a 
comprehensive study and the author was careful to compare the characteristics of 
participants to non-participants; nevertheless, he cautioned that unmeasured motivation 
might still have influenced both enrollment and program effects. Further, the number of 
students who could be served was limited by the available funds. At the time of the 
evaluation, SSS expenditures per student were falling on a year-to-year basis. Finally, the 
evaluation did not focus particularly on community colleges. Indeed, only one of the five 
colleges used in a follow-up benchmarking study was a community college (Muraskin, 
1997). 

Perhaps our understanding of the effectiveness of counseling may be distorted not only 
by the dearth of analyses of community colleges, but also by the influence of a four-year 
college perspective. The most effective counseling for residential four-year students may 
not be appropriate for part-time community college students with full-time jobs. Perhaps 
designs for student services have been particularly influenced by the four-year college 
experience. If so, it would be important to try to identify different types of counseling and 
advising and to take account of such differences when evaluating student services in 
community colleges. 

Learning Communities 

In the last 15 years, educators in both two- and four-year institutions have experimented 
with learning communities as a means of engaging and motivating students and thereby 
improving their outcomes. Learning communities organize instruction around themes, 
and students go through such programs as cohorts. Learning communities are designed to 
be more coherent and engaging than traditional courses, and to give students and faculty 
greater opportunities for increased intellectual interaction and shared inquiry (Knight 
2002; Tinto & Love, 1995). The learning community model is particularly interesting for 
community colleges since it is one way that these commuter institutions can engage with 
their students in a more intensive way than normally occurs in classrooms. If the student 
integration model does apply to community colleges, it would probably be 
operationalized with classroom oriented approaches such as learning communities. 

On the other hand, the design of learning communities may discourage non-traditional 
students from participating. To the extent that a learning community requires a cohort of 
students to attend several classes in group, it may be difficult for working students or 
students attending part time to participate. Since learning communities require close 
coordination among professors, they appear to be most effective when full-time regular 
faculty, rather than part-time adjuncts, are used. Full-time faculty are more likely to teach 
during the day, rather than at night when many non-traditional students attend classes, 
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however. These factors suggest that learning communities may attract the more middle- 
class, traditional-age students among those enrolled in community colleges. 15 

A review of research on the effectiveness of learning communities by the National 
Learning Communities Project at Evergreen Community College concluded that “a 
preponderance of studies indicate that learning communities strengthen student retention 
and academic achievement” (Taylor, 2003, p. iii). The authors of this review found more 
than 100 studies of learning communities. The large majority of them were unpublished 
single-institution assessments; only one-seventh were published in journals or books. 
Thus, while there is a tremendous amount of material on learning communities, much of 
it is difficult to access. Some of the reported studies base their assessment on 
comparisons between learning community and non-learning community students and 
some control for entering academic characteristics. But without knowing more about the 
process through which students are recruited to and enroll in learning communities, it is 
difficult to judge whether these controls are adequate to account for initial differences 
between learning community and comparison students. 

Not surprisingly, learning communities in four-year colleges have received much more 
attention than those in community colleges — only 32 of the 119 studies, or about one 
quarter, covered community colleges. Taylor (2003) chose 17 studies that were “deemed 
notable for the quality of the assessment study and the manner in which it was reported” 
(p. 4). Of these, only one used a community college. The review does not highlight 
particular issues associated with the design and effectiveness of this innovation at 
community colleges. 

The best known of the evaluations of community college programs is the 1997 article by 
Tinto published in the Journal of Higher Education, “Classrooms as Communities: 
Exploring the Educational Character of Student Persistence,” which discussed learning 
communities at Seattle Central Community College. Tinto found that participation in a 
learning community did increase the probability of quarter-to-quarter persistence. Based 
on a qualitative analysis at Seattle, he argued that learning communities promote 
persistence by facilitating the creation of supportive peer groups among students, 
encouraging shared learning, and giving students the opportunity to actively participate in 
knowledge creation. This study used a multivariate methodology that controlled for 
possibly confounding characteristics. Tinto also recognized the potential distortion in the 
model caused by student self-selection and he presented an argument for explaining why 
it was not a problem. 16 In contrast, a study of learning communities at LaGuardia 
Community College by Tinto and Love (1995) found that participation in learning 
communities did not significantly increase the probability of persistence. 

In 2003, the research organization, MDRC, started an evaluation of learning communities 
at Kingsborough Community College. The study involves freshmen between age 17 and 
34, most of whom had applied directly to Kingsborough after missing the City University 



15 We thank Kate Shaw for this point. 

16 Although Tinto addressed the selection issue, he did not provide enough information to make a judgment 
about the extent to which it is a problem. 



38 




1 7 

of New York (CUNY) system deadline. Other students who had low placement test 
scores were also invited to participate. Random assignment was used to allocate these 
students to learning communities or to a control group that received services generally 
available to students at the college. The course taking patterns of both groups were 
tracked through the second semester. Preliminary and unpublished results suggest that the 
learning community students had passed more courses (including basic English classes) 
and had higher grade point averages than the comparison group, with differences that 
were statistically significant. There were other encouraging, although not statistically 
significant, differences. 18 

Learning communities are an attractive strategy with some encouraging empirical 
support, although researchers still need to pay more attention to the process through 
which students are recruited and enroll. Learning communities offer the potential for 
more in-class opportunity for engagement with commuter students who may not have a 
chance to participate in social and other extracurricular activities at the college. 
Nevertheless, the particular problems associated with implementing learning 
communities at community colleges have not been adequately studied. Thus, practitioners 
and researchers need to make sure that the schedule and format of the programs are 
convenient for non-traditional students. This is another case where a strategy based on a 
four-year residential higher education model may need to be adjusted for the realities of 
the community college student body. 

Developmental Education and Services for Academically Under-prepared Students 

As Summers (2003) pointed out, “[M]any institutions’ primary strategy for reducing 
attrition is the early identification of students likely to drop out and the development and 
implementation of intervention services for those students” (p. 64). Therefore, colleges 
offer a variety of services for students with weak academic skills. Based on their review 
of the literature on academic achievement, Pascarella and Terenzini (1991) suggested that 
institutions can actively facilitate the academic adjustment of poorly prepared students by 
providing extensive instruction in academic skills, advising, counseling, and 
comprehensive support services. They contended that their findings have been replicated 
in several national studies, and hold even after controlling for important student and 
institutional characteristics, although they reported findings primarily from studies of 
four-year colleges. 

Early intervention for academically weak community college students, through 
counseling or other student support services, could improve their persistence and 
academic performance (Grubb, 2003; Summers, 2003). In several cases, early 
intervention has proven to be successful at encouraging students to remain in college. 
Summers (2003), in a review of the literature on the impact of counseling on attrition, 



17 Kingsborough institutional research indicated that these types of students are often the least prepared and 
have low retention rates. 

18 Personal communication with Thomas Brock, the principal investigator of the MDRC study. This study 
is scheduled to be published in early 2005 after results from a second cohort of Kingsborough students 
become available. 
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indicated that studies have found that counseling increases retention of students who are 
identified as highly likely to drop out. 

Developmental class work is the most co mm on service for students who arrive at 
community college judged to be unprepared for college-level work. Results of existing 
studies on this practice are ambiguous. Many find that students who enroll initially in 
developmental courses graduate at lower rates than students who start in regular credit 
courses (Muraskin & Wilner, 2004), although, once again, many of these studies 
analyzed students at four-year colleges. In a study of college transcripts, Adelman (1998) 
found that the more remedial courses that students are required to take, the less likely 
they are to earn a degree. Among students who attended two-year and/or four-year 
institutions and earned more than ten credits, 45 percent of those who took two remedial 
courses earned either an associate or bachelor’s degree by the time they were 30 years 
old, compared with 60 percent of those who took no remedial courses. Students who are 
judged to have low reading skills in particular are more likely to need extensive 
remediation and less likely to earn a degree. Students in the Community College of 
Denver (CCD) who started college in developmental classes were found to graduate at 
the same rate as students who started in regular courses (Roueche et al., 2001). 

Researchers studying the effectiveness of developmental education face particularly 
serious methodological challenges. On average, students who attend developmental 
education classes start out with weaker academic skills. As a result, it is hard to identify a 
causal relationship between remedial education and subsequent educational attainment. 
Even if students who start in developmental classes appear to do poorer than other 
students, it is still possible that the remediation was effective — without it, the students 
might have done even worse. Comparing similar students who do and do not participate 
in developmental education is difficult since many states and individual institutions 
mandate remediation for students with low assessment scores. 

A study by Bettinger and Long (2004) is of interest because it uses a statistical technique 
designed to identify the causal relationship in these types of cases, although, once again, 
it is a study restricted to four-year institutions. Their study focused on a large sample of 
first-time, full-time students of traditional age enrolled in Ohio’s non-selective public 
four-year colleges in the fall of 1998. Their results suggest that students placed in 
remediation are more likely to withdraw from college, but they also found that 
remediation does not seem to decrease the likelihood of transferring to more selective 
institutions or attaining a bachelor’s degree. 

There are some studies that compare outcomes for different types of developmental 
programs. The best known of these types of studies is the National Study of 
Developmental Education, conducted by the National Center for Developmental 
Education (Boylan, Bliss, & Bonham, 1997). The programmatic implications of this and 
other studies of developmental education were subsequently published in What Works: 
Research-Based Best Practices in Developmental Education (Boylan, 2002). 19 This study 



19 Since this study only examined students placed in developmental programs or courses, it could not 
measure the effect of developmental programs compared with placement into mainstream courses. 
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tested the relative effectiveness of centralized programs (compared with decentralized 
programs); programs with tutorial services with trained tutors; programs with advising 
and counseling; and those that included program evaluation on outcome measures: first- 
term GPA, cumulative GPA, retention in development courses, and success (earning a D 
or better) in math and English developmental courses. In general, this study found more 
positive results for four-year colleges than for two-year colleges, perhaps because four- 
year students may have had stronger initial skills. The study did find some positive and 
statistically significant effects for community colleges on some of these outcomes listed 
above. 

The National Study (Boylan et al., 1997) has produced a significant amount of useful 
information about developmental education and its recommendations are reasonable 
conclusions that are in accord with the experience of many practitioners. However, there 
is information not included in the published reports that would also be helpful for 
evaluating the effectiveness of remediation. For example, it would be interesting to know 
the magnitude of the effects. Also, each program characteristic is analyzed separately, so 
if there is any relationship among program components, the methodology cannot identify 
which one has an effect or whether or not a feature is effective only in combination with 
other features. Finally, it would also be important to know whether there are any other 
institutional features such as size, college organization, financial condition, or typical 
student characteristics that might influence student outcomes and also be related to a 
particular program feature. 

Boylan (2002) lists 33 features that he suggests are “best practices.” The book references 
five institutions judged to have successful developmental programs for benchmarking 
purposes, but the evidence for the 33 practices comes primarily from other studies and 
reports rather than from the five chosen sites. 21 As is the case with most of the research 
on the effectiveness of learning communities, much of the research in this study on 
developmental/remedial education is either based on unpublished documents or material 
in the National Center for Developmental Education’s own publications. Thus, while 
many of these practices are widely used and considered effective by practitioners, it is 
difficult to judge the definitiveness of the evidence supporting them. 

Given the pervasiveness of development education and the controversy surrounding the 
practice, it is surprising that there is still so much uncertainty about the most effective 
approaches to working with students with weak academic skills. Grubb (2001), in his 
review of research on developmental education, argued that learning community formats 
for developmental education did appear to have positive benefits, but he found less 
evidence on the effectiveness of other approaches to remedial education. To be sure, the 
research is complicated by the wide diversity of state regulations and locally defined cut 
scores on assessment tests and the resulting variation in the criteria used to place students 
in developmental programs. Further, many colleges use informal processes to alter these 



20 This type of information is not provided in the reports on the study. 

2 'The study itself does not present evidence of the effectiveness of the developmental programs at the five 
chosen sites. Indeed, the sites were chosen not on the basis of their outcomes (which are not reported), but 
on the basis of the programs that they used. 
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regulations on a case by case basis (Perin, in press). But this overall complexity also 
offers researchers opportunities to make comparisons among similar students who do and 
do not enroll in formal remediation or who enroll in different types of developmental 
courses. As more detailed transcript-based data become available, considerable progress 
can be made on understanding the characteristics and effects of these strategies. 

College-Wide Reform 

So far we have reported on research on individual programs. Many practitioners believe 
that the programs that they use are successful for the students who take part in them. 
However, we have suggested that an objective look at the empirical evidence and the 
methodologies used to test the effectiveness of these programs presents a more 
ambiguous picture, although there are certainly positive indications for counseling, 
advising, and learning communities. But while many people believe in the effectiveness 
of individual programs, there is much more pessimism about whether they can be “taken 
to scale.” As we have seen earlier in this report, completion rates at community colleges 
are extremely low. Making significant improvements will probably require both the 
successful expansion of pilot programs and the strengthening of related programs and 
services. No program, however well designed, can work in isolation. An excellent 
developmental or counseling program in a college with generally ineffective teaching 
may ultimately have no effect on student completion rates. We have found virtually no 
research that attempts to define and assess program institutionalization or broader 
college-wide reforms. (Later in this report we look at a measure of institutional 
completion and begin to ask what broad institutional characteristics are related to college- 
wide completion.) 

There have been initiatives designed especially to bring about reform throughout an 
institution. The best known initiative for community colleges is the Learning College 
movement. Published work (see O’Bannion, 1997) on this model presents useful 
accounts of the processes through which the colleges have brought about important 
changes, but so far no rigorous assessment of this strategy has been published. 

While community college practitioners believe that few colleges have been able to bring 
pilot programs to scale successfully, many are convinced that the Community College of 
Denver (CCD) has, over the last 20 years, succeeded in bringing about fundamental 
reforms in the basic ways that the college operates (Roueche et al., 2001) These reforms 
followed a systematic planning and benchmarking process and have included major 
changes in organization, teaching methods, counseling and student services, relationships 
to the community, and organizational philosophy. Significantly, the college also 
emphasizes judging success on the basis of measured outcomes. Roueche et al. (2001) 
also report increases in the number of graduates and in the number of graduates of color 
and the conclusion that “cohort tracking indicated no significant difference in student 
success on the basis of race, ethnicity, age, or gender” (p. 23). They report that many 
CCD students of color transferred successfully, that there were high rates of student 
satisfaction with programs and instructors, and that surveyed employers demonstrated 
unanimous satisfaction with the skills of CCD graduates. These are encouraging results, 
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although questions remain about the implications and interpretations of the numbers. 
Given the importance of the CCD case, further investigation is warranted. Careful 
investigations of other institutional change efforts will also be important. Nevertheless, 
the CCD experience suggests that institution- wide reform can have an effect and that 
perhaps a focus on individual programs may be less effective. 

Conclusions 

Thus, we are left with a good deal of research that has generally positive results for first 
year experiences, learning communities, supplemental academic services, advising, and 
counseling. Results from the evaluation of student support services are encouraging and 
research on learning communities have generally positive conclusions. Our knowledge 
about the design of remediation remains ambiguous and, consequently, controversial 
methodological problems are particularly important here. Certainly we need to strengthen 
our understanding of this fundamental community college activity. 

More attention needs to be paid to how programs can be combined and what factors 
promote the institutionalization of successful pilot programs. Combining different 
reforms into comprehensive college-wide initiatives remains attractive, but we are far 
from any concrete understanding about how programs should be combined. Further, 
moving a successful pilot program to college -wide scale may be more difficult than 
implementing the pilot. So far, research has offered little guidance on these issues. 

Since most of the programs that we have examined are voluntary, comparisons of 
outcomes for participants and non-participants need to be interpreted carefully. Some 
analysts have done a better job than others of taking into account potentially confounding 
factors. In almost all cases, however, research needs to pay more attention to the program 
recruitment and enrollment processes. 

Research on four-year colleges continues to dominate the field, although this emphasis 
might be beginning to change. Conceptual models that are probably more appropriate for 
four-year environments still have a strong influence, and efforts to develop theories more 
suited to community colleges need to accelerate. The influential student integration 
model implies that colleges should work towards engaging their students both 
academically and socially, but the empirical support for this perspective is much weaker 
for community colleges than for four-year colleges. One promising approach for 
community colleges might be to design services based on the assumption that part-time, 
working students will have a much weaker social and psychological attachment to the 
institution. The focus on learning communities, which attempt to engage the students in 
the classroom, is one implication of this perspective, although its format, too, may be 
most effective for more traditional students. 



22 For example, the levels of satisfaction are reported for one point in time but the reader does not know 
what those rates were before the introduction of the college-wide reforms. Enrollments and graduation rates 
improved but were there any accompanying changes in the skills of entering students? How was “student 
success” defined and what control variables were used in the cohort analysis? 
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Certainly one of the most encouraging current trends is the growing emphasis on analysis 
of student outcomes at both the state and the local level. Statewide data systems and more 
sophisticated cohort tracking on the campuses offer many opportunities for useful 
analysis. It is possible that there is a larger volume of more definitive research than is 
evident from a review of published work, since many studies are unpublished, published 
in obscure outlets, or published without providing enough information to judge the 
validity of their conclusions. More energetic and comprehensive efforts to disseminate 
and discuss assessments may reveal more useful findings. 

Thus, there is still much work to do to strengthen the research and assessment carried out 
on the campuses and in state capitals and to disseminate the research that could be 
available. But there is not necessarily widespread agreement about what constitutes 
useful analysis. In giving colleges advice about how to conduct evaluations on 
developmental education, one influential expert stated: “Evaluation of developmental 
education does not require the use of complicated statistics. Program outcomes can be 
accurately described using nothing more than percentages, bar graphs, and pie charts” 
(Boylan, 2002, p. 42). We agree that these types of descriptive statistics can be a useful 
way to begin discussion and investigation, and that they are important to provide a basis 
for wide engagement of faculty and administrators in discussions of policies and 
practices. Nevertheless, evaluating the effectiveness of educational programs is extremely 
difficult and descriptive information should be seen as only a first step in understanding 
program effects. A thoughtful attempt to understand how the participants and non- 
participants compare on other factors that might influence the outcomes is essential. 

At the same time, program improvement cannot await definitive research results. 
Educators, policymakers, and students must move forward based on the best information 
available, even if that information may be subject to alternative interpretations. But as 
policymakers, private funders, and the public have increasingly turned their attention to 
community colleges, we have the opportunity to strengthen the available research and 
provide more useful information as administrators and faculty members try to improve 
the educational outcomes of their students. 



V. Institutional Characteristics Associated with 
Completion and Persistence at Community Colleges 

In this section of the report we move from an analysis of individual programs to a more 
institutional focus. Our basic strategy is to examine college measures of student 
outcomes — for example, the percentage of an entering community college cohort that 
earns an associate degree within three years. These rates vary dramatically. Our research 
project begins to illuminate why some colleges have much higher rates of degree 
completion than others. 

This analysis is the first step in a longer project designed to identify institutional 
characteristics and policies that are related to improved graduation and persistence. But 
there are many factors that might cause variation in student success and many of them 
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may be beyond the control of an individual institution. For example, a great deal of 
research suggests that students who attend part time, those from lower income families, 
and those with lower academic skills tend to complete at lower rates. Thus, two colleges 
that are equally effective may have different graduation rates if the one with a lower rate 
enrolls students from lower income families. Therefore, any research that tries to relate 
practices to outcomes must take into account other factors that impact graduation rates. 

These factors must also be taken into account when graduation rates are used for 
purposes of accountability — that is, to judge the performance of an individual college. 
Failing to control for students’ academic readiness and other characteristics unfairly 
penalizes institutions, such as community colleges, that enroll less well-prepared students 
and gives undeserved credit to those with selective admissions policies. 

The research reported here uses institution-level data available from the Integrated 
Postsecondary Education Data System (IPEDS) to explain graduation rates. These data 
include student characteristics; institutional characteristics, such as location and 
enrollment size; and some financial characteristics, including expenditures for various 
types of services. The analysis will therefore help demonstrate how these factors are 
related to student outcomes. Even after taking the factors into account, wide variations in 
institutional completion rates remain. The next step in this research program, then, will be 
to use a variety of quantitative and qualitative methods to try to identify the institutional 
policies that explain the remaining variation. 

We first review existing literature that analyzes institutional graduation rates, 
summarizing the characteristics that are most strongly related to these student outcomes. 
We then present our own research on community college graduation rates. Finally, we 
show how these results can be used at both the state and institutional levels as a 
foundation for further exploration of practices and policies that are associated with higher 
levels of student success. 

Literature Review 

A handful of studies have carried out this type of institutional analysis, although not 
surprisingly all of these studies have focused on four-year colleges. For example, in one 
study Porter (2000) found that average SAT scores and the percent of students who are 
female were associated with higher graduation rates, while institutions with higher 
percentages of students over age 25 tended to have lower graduation rates. Some more 
structural characteristics, such as expenditures per student, smaller total enrollment, and 
the availability of on-campus housing, were also related to higher graduation rates. Porter 
showed that performance measures such as graduation rates are sensitive to the variables 
used in the analysis and to the specification of the model. His research also highlights a 
serious methodological problem with this type of research. For example, one might 
conclude that building housing for students might increase graduation rates, but it is also 
possible that the availability of housing attracts a particular type of student who would 
have a higher likelihood of graduating, with or without the housing. Thus, housing might 
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increase a college graduation rate without increasing the probability that any particular 
student would graduate. 

Astin, Tsui, and Avalos (1996) used data on a nationally representative sample of first- 
time, full-time students to compare the graduation rates of 365 four-year institutions in 
1985. They found that private universities had the highest graduation rates, and believed 
that this finding resulted from the fact that such institutions tend to enroll better prepared 
students. They also found that highly selective institutions and those that enroll large 
numbers of students in fields like business, psychology, and the social sciences have 
higher graduation rates. Institutions with large engineering programs, commuter schools, 
and larger colleges have lower than expected rates. Mortenson (1997) also used a 
regression model to estimate predicted graduation rates for 1,100 baccalaureate-granting 
colleges. He had similar findings to Astin et al. (1996), namely that institutions whose 
students have higher average SAT scores and those with a higher percentage of freshmen 
living on campus had higher graduation rates, while those with many part-time students 
and relatively large engineering programs had lower rates. 

Ryan (2003) used data on 363 four-year institutions to estimate the impact of institutional 
expenditures for instruction, academic support, student services, and administrative 
support on the six-year graduation rates of cohorts within each institution. He controlled 
for student SAT/ACT and other factors. His findings suggest that instructional and 
academic support expenditures have positive and significant effects on cohort graduation 
rates, as previously indicated by Astin (1993). However, expenditures on student services 
and expenditures on administrative (institutional) support failed to produce any 
significant impact on graduation rates. Ryan suggested that there are trade-offs in the 
utilization of financial resources within an institution in terms of degree attainment, and 
that institutions should be careful when deciding where to allocate resources. 

More recently, Scott, Bailey, and Kienzl (in press) conducted an analysis of six-year 
graduation rates for baccalaureate-granting institutions based on data from the College 
Board’s American Survey of Colleges and from IPEDS. They used grouped logistic 
regression, arguing that the ordinary least squares method used by previous researchers is 
not an appropriate technique since graduation rates are constrained within an interval 
ranging from 0 to 100. Using this new methodology, they found that private colleges and 
those with students with higher average SAT scores, a higher proportion of women, and 
higher instructional expenditures per FTE student had higher graduation rates. 

Institutions with higher proportions of minority students, older students, and part-time 
students had lower graduation rates. In addition, they found that institutions with higher 
in-state tuition also tended to have higher graduation rates, even after controlling for 
student characteristics. 

These studies from the last 10 years roughly confirm the findings summarized in 1991 by 
Pascarella and Terenzini (1991), who also found that four-year colleges with students 
with higher SAT scores, students from higher income families, and more full-time and 
female students had higher graduation rates. Private and residential institutions also had 
higher rates. These findings are consistent with the implications of Tinto’s integration 
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model. Pascarella and Terenzini (1991) also reported interesting findings on historically 
Black and women’s colleges. African-American students enrolling in predominantly 
White colleges are more likely to dropout than are African-American students who attend 
a Historically Black College or University (HBCU). They hypothesized that African- 
American students experience higher levels of social integration in HBCUs than in 
majority White institutions. They used the same reasoning to argue that attendance at a 
single-sex institution increases the level of persistence and educational attainment of 
women; although studies indicate that the magnitude of this effect is quite small. 

Table 5 summarizes the institutional characteristics that research has found to be 
associated with attainment by undergraduate students. Again, most of this research has 
been conducted on four-year institutions. 



Table 5. Institutional Characteristics Associated with Bachelor’s Degree 
Completion 



Institutional Characteristic 


Effect on Degree 
Completion 


More Instructional expenditures 


+ 


Greater Selectivity 


+ 


Average student characteristics 




• Larger average student family income and parents’ education 


+ 


• Higher percent full-time students 




• Higher percent female 


+ 

+ 



What can community colleges learn from the findings at four-year institutions? Assuming 
that the factors analyzed in these studies will have similar effects for community colleges, 
then the problems that these colleges face in increasing their graduation rates are clear. 
The studies consistently find that the characteristics of students that predict lower 
graduation rates are precisely those that community colleges possess in greater 
abundance relative to four-year institutions. Thus, it suggests that colleges must reduce 
their numbers of such students. But by attempting to improve graduation rates by 
becoming more selective, community colleges would violate one of their underlying 
missions — that of open access for all. Thus, attempting to demographically appear more 
like four-year colleges would be to the detriment of the colleges and their communities. 

The studies we reviewed also have important, though disturbing findings about financial 
variables. The consistent finding in all of these studies is that higher instructional 
expenditures yield higher completion rates, even after controlling for student 
characteristics. This finding does not bode well for community colleges in the current 
political and fiscal environment, where, over the long run, per-student state support will 
probably continue to fall. Since higher tuition at four-year colleges does appear to 
promote completion, colleges might compensate for lower state funding by increasing 
tuition and thereby preserving completion rates. Unfortunately, high tuition also violates 
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the open access mission of community colleges. However, we suspect that the higher 
tuition effect may actually result from student selection and would probably disappear if 
there were better measures of student family income. In any case, in contrast to the 
findings for four-year colleges, our research on community colleges (discussed below) 
actually finds that higher tuition is negatively related to completion. 

Analysis of Institutional Graduation Rates at Community Colleges 

In this section of the report, we conduct an analysis of institutional graduation rates and 
their determinants at community colleges using IPEDS data. The National Center for 
Education Statistics publishes three-year degree completion rates for community colleges 
in its Graduation Rate Survey (GRS) — often referred to as the Student Right-to-Know 
(SRK) data, which is part of IPEDS. The “degree completion rate” refers to the 
percentage of first-time, full-time students who entered a community college in a 
particular year and received a certificate or associate degree from that same institution 
within three years, or in 150 percent of the time traditionally associated with earning a 
two-year college degree. Colleges exhibit wide variation in graduation rates, and in this 
section, we exploit this variation to determine the institutional characteristics that are 
related to higher institutional completion rates. 

Using the GRS we identified the actual degree completion rates of colleges. Then, using 
additional data available in IPEDS, we computed a “predicted” graduation rate. This is 
the graduation rate that we would expect a college to have, given its institutional 
characteristics. We then compared actual and predicted rates. When the actual rate was 
higher than the predicted rate, we concluded that the college is, in effect, performing at a 
higher rate than we would expect. The converse would be true for colleges with lower 
than predicted rates. 

We first summarize the latest data from IPEDS on community college graduation rates. 
We then present a model for using IPEDS data to estimate a predicated graduation rate 
for community colleges, controlling for student characteristics, resources, and other 
factors that affect student outcomes. We conclude with a discussion of how an index 
created by comparing the predicted graduation rate with the actual rate reported by each 
college in IPEDS can be used for further research on institutional policies and practices 
that can enhance outcomes for low-income and minority students. 

Community College Graduation Rates 

Figure 8 presents data on community college outcomes from the 2002-03 IPEDS 
Graduation Rate Survey. The survey population for this dataset includes all first-time, 
full-time (FTFT), degree- or certificate-seeking undergraduate students entering the 
institution either during the fall term (by October 15) of the 1999-2000 school year. 23 



23 Institutions that do not offer programs based on standard academic terms use the 12-month period (Sept. 
1 - Aug. 31) to determine their cohort. This description comes from the National Center for Education 
Statistics “Instructions for Graduation Rates - 2-year Institutions.” See 
http://nces.ed.gov/ipeds/pdf/webbase2002/grs_2yr_form.pdf 
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These completion rates are institution- specific measures in that they only capture 
outcomes for students who begin and complete in a particular college. They do not 
capture students who start at a given college and move to another institution before 
earning a degree. 



Figure 8: 1999-2000 First-Time, Full-Time Students: Three-year Community 

College Graduation Rates by Race/Ethnicity and Gender. 
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Source: IPEDS Graduation Rate Survey 2002-03. 



Overall, 22.3 percent of FTFT community college students in the sample attained a 
postsecondary credential in their starting institutions after three years. These findings are 
consistent with other results from the research literature. As the figure shows, graduation 
rates for women are higher than for men; and all minority populations graduate at lower 
rates than do Whites. Observe that contrary to findings at four-year institutions, Asian 
students in community colleges are also less likely to earn a degree or certificate than are 
Whites. 

Data 

To better understand the institutional characteristics associated with the graduation rates 
reported above, we used IPEDS data to create three statistical models of the three-year 
institutional graduation rates for cohorts of FTFT community college students in 1999- 
2000. 
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Sample 



IPEDS includes information about the entire population of higher education institutions 
in the United States and its outlying areas. In order to extract a community college 
sample, we first considered only public two-year institutions in the United States. We 
also excluded institutions without regional accreditation and those that are non-degree- 
granting (granting only certificates). Finally, we used the procedure recommended by 
Hadi (1992; 1994) to identify outliers — institutions with improbably high values for 
institutional variables. Using this method we eliminated three institutions with outlier 
values for instructional expenditures. 24 Thus, the final sample contains 915 community 
colleges. 

Outcome Variables 

We used the overall degree completion rate for each institution from the 2002-03 IPEDS 
Graduation Rate Survey. Time-limited completion rates at community colleges are 
lowered both because most students attend part-time (and therefore take longer to 
complete, even if they eventually do complete) and because many may not be seeking 
degrees (they may enroll in a limited number of courses in order to advance their job 
skills or pursue a personal interest). The dependent variable used here measures 
completion rates only among first-time students who were enrolled full time and were 
seeking a credential. First-time, full-time students represent only a minority of 
community college students — 17 percent of community college FTE enrollments in 
1999-2000, according to IPEDS. This means that the measure used here, based on full- 
time, credential-seeking students, likely overestimates the graduation rate for most 
community colleges. In any case, as is indicated below, we controlled for the proportion 
of a college’s students who are enrolled part time. 

Our dependent variable does have limitations. It measures degree completion by students 
at their first postsecondary institution and does not capture students who complete 
programs at other institutions. Also, IPEDS does not provide data to break out the degree 
completion measure by type of credential awarded. Since graduation rates for certificate 
programs are higher than they are for associate programs (Bailey, Alfonso, Scott, & 
Leinbach, in press), institutions such as technical colleges that specialize in awarding 
certificates may have higher overall graduation rates than comprehensive community 
colleges, which are relatively more likely to emphasize programs leading to associate 
degrees. To account for this fact, we include a dummy variable indicating whether an 
institution awards more certificates than associate degrees. 



24 These community colleges are Ilisagvik College (AK), College of the Marshall Islands (MH), and Los 
Angeles County College of Nurse and Allied Health (CA). We made some other adjustments in the sample. 
Missing values in the variable “Total certificates awarded” have a differential treatment. Our sample 
contains 43 institutions with missing values in this variable and we checked the accuracy of this 
information on their web pages. For those cases where the community college does not award certificates, 
we impute a zero instead of a missing value. Finally, changes were made for 21 institutions, which are 
available from the authors on request. 
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Explanatory Variables 



We selected the explanatory variables for the model based on factors that previous 
studies have indicated are related to degree completion in community colleges. The 
inclusion of most such variables are self-explanatory. However, in addition, we included 
dummy variables for individual states with at least two community colleges. States, 
through their distinct policies and financial levers, can have fundamental influence on the 
graduation rates of colleges. Consequently, community colleges may play very different 
roles within the overall systems of higher education in different states. For example, 
Florida has a statewide articulation agreement that guarantees admission to a public four- 
year institution to any student who earns an associate degree. This gives students in 
Florida a strong incentive to complete their degree. While the significance of any 
individual state dummy variable does not in itself explain graduation rate variations, the 
assumption is that some set of practices of that state is having an impact. We expect that 
the statistical findings from the models will lead us to investigate policies of those states 
with significant state variables. 

We grouped our explanatory variables into three categories: Fixed Institutional 
Characteristics (mostly geographic variables), College Characteristics (mostly 
characteristics of the student population), and Financial Characteristics. Table 6 lists the 
variables by category along with the expected effect (positive or negative) of each. 



51 




Table 6. 



Explanatory Variables in a Model of Community College Graduation 
Rates 



Explanatory Variable 


Hypothesized Effect 


Fixed Institutional Characteristics 




Urban 


- 


Suburban [reference group] 
Rural 




State dummies 


Historically Black College or University or tribal college 


- 


College Characteristics 


Size (1999 full-time equivalent undergraduates) 
0-1000 FTE [reference group] 

1001-2500 FTE 




2501-5000 FTE 


- 


5000+ FTE 


- 


Racial/ethnic makeup of students 




Percent FTE White [reference group] 
Percent FTE African-American 




Percent FTE Hispanic 
Percent FTE Asian 


+ 


Percent FTE Native American 


- 


Percent FTE part time 


- 


Gender (percent FTE female) 


+ 


College grants more certificates than associate degrees 


+ 


Financial Characteristics 




Average in-state tuition 


- 


Instructional expenditures per FTE undergraduates 


+ 


Academic support per FTE undergraduate 


+ 


Student services per FTE undergraduate 


+ 


Federal aid per FTE undergraduate 


- 



Our list of explanatory variables has some key omissions. As discussed, studies of the 
graduation rates of four-year colleges indicate that institutions that enroll better-prepared 
students, as measured by SAT/ACT scores, tend to have higher graduation rates. 
Unfortunately, the IPEDS dataset does not include data on the academic preparation or 
the readiness of entering students. In general, there is not a widely used and available 
measure of the academic preparedness of community college students. Moreover, the 
IPEDS dataset does not include information on the income characteristics of students. We 
used the amount of federal aid per FTE undergraduate as a crude proxy for the extent of 
financial need among a college’s students, although we realize that this measure is 
problematic. For example, an institution might serve many low-income students, but still 
receive relatively little funding through federal financial aid because it is prohibited from 
doing so due to high default rates or, perhaps, because it does not encourage and assist 
students in applying for aid. 



52 



We created these variables from four IPEDS surveys for 1999-2000 (the base year for the 
cohorts of students for which we have GRS data): Institutional Characteristics, Fall 
Enrollment, Finance, and Completions. Table 7 gives the mean values from IPEDS of 
each variable in the model for the community colleges in the sample. This provides us 
with the characteristics of the “average” college in the sample. For example, observe that 
most of the community colleges are located in suburban areas. The average school enrolls 
3,044 full-time equivalent (FTE) students, most of whom are female (57.5 percent), and 
charges $1,659 for in-state tuition per academic year. Nearly 12 percent of FTE 
undergraduates are African-American and 8.1 percent are Hispanic. One in five colleges 
awards more certificates than associate degrees. 
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Table 7. 



Descriptive Statistics of the Community College Sample 



Variable 


Mean 


Standard 

Errors 


Cohort 


Full-time, full-year (FTFY) degree-seeking undergraduates 


504.689 


494.177 


Number who attained a degree within three years 


112.639 


123.948 


Fixed Institutional Characteristics 


College is located in urban area 


0.387 


0.487 


College is located in suburban area [reference group] 


0.524 


0.500 


College is located in rural area 


0.090 


0.286 


College is an Historically Black College or University 


0.009 


0.093 


College is a tribal college 


0.019 


0.135 


College Characteristics 


Total full-time equivalent (FTE) undergraduates [1000] 


3.044 


2.927 


1000 FTE undergraduates or less [reference group] 


0.207 


0.405 


1001-2500 FTE undergraduates 


0.370 


0.483 


2501-5000 FTE undergraduates 


0.250 


0.433 


More than 5000 FTE undergraduates 


0.173 


0.378 


Institution awards more certificates than associates 


0.171 


0.377 


Percent FTE African-American undergraduates 


0.121 


0.157 


Percent FTE native American undergraduates 


0.031 


0.123 


Percent FTE Asian undergraduates 


0.033 


0.078 


Percent FTE Hispanic undergraduates 


0.081 


0.143 


Percent FTE White undergraduates [reference group] 


0.689 


0.241 


Percent FTE part-time undergraduates 


0.339 


0.132 


Percent FTE female undergraduates 


0.575 


0.074 


Financial Characteristics 


Average in-state tuition [$1000] 


1.659 


1.184 


Instructional expenditures per FTE undergraduate [$1000] 


4.157 


1.491 


Academic support per FTE undergraduate [$1000] 


0.817 


0.695 


Student services per FTE undergraduate [$1000] 


0.981 


0.697 


Federal aid (Pell grants) per FTE undergraduate [$1000] 


0.824 


0.467 



Source: Authors’ calculations using data from IPEDS Graduation Rate Survey 2002-03. 
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Methodology 

Following Scott et al. (in press), we estimated predicted three-year graduation rates for 
the community colleges in the sample using a grouped logistic regression method. 25 This 
method allows for an aggregate analysis in the sense that the mean values of the “inputs” 
are used in the evaluation of institutional graduation rates, which are themselves the 
aggregation of individual student successes. In this study, each observation is an 
institution; however, the outcome variable is the result of the behavior of a cohort of 
students in the institution. Specifically, the dependent variable is the percentage of a 
cohort of first- time, full-time students in 1999-2000 who completed a certificate or 
degree program at the starting institution by 2002-03 — in three years or 150 percent of 
the conventional time. 26 

Findings 

The Appendix presents the results of grouped logistic regressions of three sequential 
models when the outcome variable is degree completion. The group logit coefficients 
have no straightforward interpretations. Therefore, in the Appendix, we compute the 
marginal effect for each variable to make the results easier to interpret. Thus, the 
marginal effect represents the change in the completion rate of a one unit change in the 
characteristic, holding all other characteristics constant. 

Model One 

The first model adds only geographic-related variables, such as degree of urbanicity 
(urban and rural) and 5 1 state dummies (including the District of Columbia) to control for 
state differences. In general, colleges located in urban areas are predicted to have nearly 4 
percent lower graduation rates, while rural colleges can expect 3 percent higher 
completion rates. Historically Black community colleges and tribal colleges do not 
perform significantly different from other institutions. With just these geographic 
measures, the model is explaining roughly 35 percent of the variation in institutional 
graduation rates. 



25 The institutional graduation rate, while a continuous variable, is constrained to lie within 0 and 1. This 
procedure produces more robust results than does the ordinary least squares (OLS) method applied in most 
previous studies (see for example, Mortenson, 1997). By definition, the framework implies a 
heteroskedastic regression format, which can be estimated using weighted least squares (see Greene, 2003, 
pp. 686-689, for details). In the first step, OLS regression is used to estimate consistent but inefficient 
coefficients and then results feed into the corrected variances to estimate the second step in the weighted 
least square regression (routines like glogit in STATA™ estimate these procedures). 

26 Using grouped information has clear limitations: the method explains institutional characteristics 
associated with completions rates, while the behavior of each individual in each cohort is not explicitly 
taken into account. Moreover, the model assumes that each individual in a given institution has the same 
probability of graduation. In another words, the method assumes that each cohort member is affected by the 
geographic, college, and finance characteristics in a similar way. 
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Model Two 



The second model adds dummies for enrollment size, as well as the racial/ethnic and 
gender composition of the campus. Size is an important predictor of an institution’s 
degree completion rate: larger community colleges, especially those with more than 
2,500 FTE undergraduates, have 8 to 1 1 percent lower graduation rates than do smaller 
colleges. This result remains even after controlling for college location. Being in an urban 
area remains negative but now becomes insignificant. 

In terms of the demographic characteristics of the student body, having a large percentage 
of African-American, Asian, or Hispanic students enrolled is negatively associated with 
an institution’s degree completion rate. Colleges with a relatively larger part-time student 
population have lower completion rates (even for full-time students). As expected, 
colleges that award more certificates than associate degrees have slightly higher rates of 
degree completion. 

The relationship between graduation rates and the share of women in the student body is 
surprising, and quite unexpected. 27 Almost all research on retention and graduation shows 
that women graduate at higher rates than men, even after controlling for other 
demographic characteristics. Indeed, even in GRS, first-time full-time female students 
have higher graduation rates than comparable men. Nevertheless, we found that colleges 
with more female students tend to have lower graduation rates. Of course it is possible, 
although still surprising, for women to graduate at higher rates while the full-time 
graduation rate of the college is lower. After all, the GRS sample only includes on 
average 17 percent of the FTE student population. Moreover, the negative female effect 
remains even when male and female rates are analyzed separately and when using a 
variety of different specifications. Clearly this result needs further study. 

It is also important at this point to explain the meaning of our coefficients. Given that we 
are dealing with institution-level data, the interpretation of these variables represents the 
effect of campus or other environmental factors on the likelihood of a FTFT community 
college student in our cohort to achieve a positive educational outcome — and not the 
likelihood of a particular individual in a particular racial/ethnic group to attain a degree. 
For instance, having a large percentage of minority students enrolled at a college lowers 
the probability of the completion of any FTFT student even after controlling for other 
characteristics of the college. In the same way, having a substantial percentage of female 
and part-time undergraduates affects the completion likelihood in a similarly negative 
way. 



27 As mentioned above, caution must be taken when interpreting the marginal effects. For instance, the 
marginal effect on the “percent FTE female” variable is -50 percent. This does not mean that every percent 
change in female enrollment lowers the institution’s graduation rate by 50 percent. Since the variable is in 
the same units as a binary variable, the marginal effect represents the change in graduation rate between a 
community college with no females and a similar college that is female only. 
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Model Three 



The final model adds finance and financial-aid related variables. In-state tuition, 
instructional expenditures, and student services are significant, but the magnitude of the 
effect is not very large. Community colleges with higher than average in-state tuition 
have lower degree completion rates. On the other hand, community colleges that invest 
relatively more in instruction have higher rates of degree completion. An additional 
$1,000 spent on instruction per FTE undergraduate improves graduation rates by 1.2 
percent. Finally, a higher level of expenditure in student services is associated with lower 
levels of graduation rates. The effect of the variables remains similar to what was found 
using the previous models, and the model fit does not improve greatly with the addition 
of the finance-related measures — around 60 percent of the variation in the outcome is 
explained by the model. 

Conclusion 

This analysis confirms several hypotheses about institutional determinants of graduation 
rates at community colleges, but also raises some questions. Our study, like others, found 
that colleges with many part-time students have lower graduation rates, even for the full- 
time students. Many studies reveal lower graduation rates for African-American and 
Hispanic students and our results are consistent with them as well. Other research shows 
an ambiguous relationship between school size and graduation but our analysis indicates 
a consistent negative relationship between a large enrollment and completion. Colleges 
that emphasize certificates, not surprisingly, have higher completion rates, but this is an 
artifact of the measure — certificates and associate degrees are combined without 
differentiation in the graduation rate. Higher tuition is related to lower graduation, while 
greater instructional expenditure is related to higher graduation. Determining the 
direction of causality in a cross-section analysis is always difficult, but these results do 
suggest that the current trend towards higher tuition may lead to lower completion, 
although if funds are used to increase instructional expenditure, the two effects may 
cancel each other out. 

Use of the Results to Benchmark Performance of Community Colleges 

Our analysis has two broad purposes. First, it can be used to identify institutional 
characteristics that are associated with higher completion rates. Second, it can identify 
colleges and states that perform at higher levels after measurable characteristics are taken 
into account. While our findings do not directly reveal the factors that cause better 
performance, they indicate where we might search for those factors. 

For the state-level analysis, we have included state dummy variables in our models. The 
coefficients on those dummies are a measure of how well colleges in the state perform on 
average, after taking account of the other characteristics in the model, relative to colleges 
in other states. California, Florida, and Nebraska are the three states with the highest 
average adjusted (for the control variables) graduation rates. Before we conclude that 
these states have found the key to improved retention, however, we must first understand 
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the institutional and legal context in these states. We have already suggested a regulatory 
reason why graduation rates in Florida might be higher: the incentive of acceptance at a 
four-year institution encourages students to earn an associate degree. The states in the 
Lumina Achieving the Dream initiative other than Florida — Texas, North Carolina, 
Virginia, and New Mexico — all have overall graduation rates below the average for 
states. We consider these state comparisons to be preliminary; nevertheless, they do 
suggest fruitful areas for further investigation. 

Our method for determining relative performance at the college level is based on 
computing the difference between the actual graduation rate and the “predicted” rate, 
which is based on the college characteristics and the coefficients calculated in our logistic 
analysis. Colleges whose actual completion rates are higher than their predicated rates do 
better than expected in enabling students to complete degrees. Conversely, colleges 
whose actual rates are lower than their predicted completion rates do not graduate 
students at a rate that would be expected, given the mix of students they serve, the 
financial resources available to them, and other factors. We identify a group of “high 
performers” as those with predicted outcomes two standard deviations higher than the 
overall predicted mean. “Low performers” are identified as having a predicted outcome 
that is more than two standard deviations lower than the overall predicted mean. 

To illustrate this approach, among the colleges participating in the Lumina Achieving the 
Dream initiative, the raw graduation rates range from 1 percent to 34 percent. Several of 
the colleges with the highest graduation rates also perform above their expected levels 
based on their characteristics. But for some, the ranking shifts significantly after 
controlling for college characteristics. The college with the fourth highest graduation rate 
(25 percent) actually has a predicted rate of 34 percent. Therefore, while this college 
ranks fourth (out of 27 colleges) based on its raw graduation rates, it ranks eighteenth 
when the difference between the expected and actual graduation rates is used. Similarly, a 
college ranked eighteenth based on its raw graduation rate rose to tenth based on its 
performance relative to its expected rate. 28 In future work, we plan to use this type of 
analysis to identify a sample of colleges that do better or worse than expected for a more 
in-depth study on which institutional policies and practices result in improved attainment 
for students. 



VI. Conclusions and Next Steps 

This report is part of a broad initiative, Achieving the Dream: Community Colleges 
Count, funded by the Lumina Foundation for Education. The initiative is designed to 
increase retention, completion, and success for low-income students, students of color, 
first generation college students, and other underserved groups at community colleges. 

The report begins with a general discussion of enrollment and completion data for 
community colleges, which provide important access to postsecondary education for a 
wide and varied range of students. The data clearly show that low-income, minority, and 



28 The community college completion benchmark index is available in a spreadsheet from CCRC. Colleges 
can use the index to benchmark their institutional completion rates against those of other institutions. 
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other underserved students are overrepresented at these colleges, and they complete 
degrees and certificates at relatively low rates. These conclusions hold even when 
multivariate methods of analysis are used. 

The rest of the report discusses community college retention and completion from various 
perspectives. First, we reviewed the arguments about whether graduation rates were an 
appropriate index against which to judge community colleges, noting, in particular, that 
some students enroll with an educational goal other than earning a degree and do, indeed, 
meet it. We next reviewed institutional practices designed to increase college completion, 
and in the final section we used institutional characteristics — such as enrollment size and 
student demographic and financial characteristics — to explain community college 
graduation rates. We used this analysis to develop an index that can be used to 
benchmark college graduation rates. 

On the basis of our findings, we present here a set of programmatic and policy 
implications, divided into three groups: (1) retention, graduation, and transfer rates; (2) 
programs and practices; and (3) research on community colleges. 

Retention, Graduation, and Transfer Rates 

(1) Colleges must recognize the need to improve retention, graduation, and 
transfer rates. Community college faculty and administrators are often defensive 
about graduation and transfer rates, arguing that there are many good reasons why 
more students do not graduate or transfer and that the critics of college graduation 
rates fail to understand the role and nature of the colleges. We acknowledge the 
validity of many of these points; nevertheless, having good explanations for why 
some students never finish a degree or certificate is not the same as saying that 
graduation rates should not be higher, and most colleges are indeed trying to 
increase retention. We have shown that there is significant variation in college 
graduation rates, even after taking account of many of the factors that are believed 
to thwart retention. Thus, some colleges have been relatively successful at 
overcoming many of the barriers that all colleges face. 

Practice and Policy 

Despite a large amount of research on retention in higher education, we have much to 
learn about the effects of institutional practices on retention and completion at 
community colleges. A search for lessons for community college practitioners is thwarted 
by an overwhelming emphasis among researchers on four-year colleges and serious 
methodological difficulties in conducting studies that would provide useful information. 
Thus while available research can provide guidance for community college practice at the 
local and state levels, practitioners should be aware that many questions remain about the 
design and effectiveness of common innovations and programs. 

(2) Current research does provide support for the effectiveness of learning 
communities. Learning communities have attracted a great deal of attention in the 
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last decade and the research on their effectiveness is generally positive. They may 
be particularly significant for community colleges, since, unlike at a residential 
college, the classroom may offer the only opportunity to engage the students with 
the institution. The learning community format also appears to be effective for 
students in developmental education. At the same time, though, in developing 
these programs it is important for colleges to work out ways to organize them to 
allow the participation of a wide range of students, including part-time students 
and those who work. 

(3) Research on counseling, advising, and student orientation suggests that all 
can be effective for retaining students, but many questions about design and 
intensity remain to be answered. Orientation and first-year seminars appear to 
be effective, but much of the relevant research concerns four-year colleges. 
Research on the TRIO Student Support Services suggests a positive effect, but 
that work is now close to a decade old. In addition, institutionalization of the 
program beyond the targeted students remains a problem. Many practitioners are 
convinced that comprehensive services that include personal and career 
counseling and academic advising are effective. Yet, many questions still need 
answers: what is the role of faculty, what should constitute the educational 
preparation of counseling and advising staff, how can services be provided 
without excessive cost, and how much counseling can be done on-line? 

(4) Tuition levels, instructional expenditures, and institution size are related to 
graduation rates. Our research suggests that higher tuition and lower 
instructional expenditures are related to lower graduation rates. Expenditures on 
student services are not related to completion rates, but the lack of an effect could 
be explained if expenditures on student services reflect a greater need for such 
services rather than an exogenous institutional policy. Our results on the effects of 
expenditures are worrisome in an environment in which state expenditures on 
community colleges are falling. Colleges with enrollments under 1 ,000, on 
average, have graduation rates about 10 percentage points higher than colleges 
with enrollments over 5,000, even after controlling for urbanicity. It is unrealistic 
to propose cutting the size of colleges, but the result here does suggest that it 
would be interesting to try to explain what smaller colleges do that might 
influence retention. Indeed, an extensive discussion of the effects of school and 
classroom size at the K-12 level has been occurring for some time. 

(5) It is essential to promote a more thorough discussion of the determinants of 
student outcomes and the effects of programs and policies on those outcomes. 

There has been a tremendous amount of discussion of effectiveness and student 
outcomes, and some reports contain descriptive numbers. Accountability-related 
reports often also contain enrollment and graduation percentages for particular 
demographic groups. Still, as we have pointed out, the published reports of 
college practices rarely contain thorough discussions or analyses of outcomes that 
address questions of causality. 
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For many community college faculty, the best opportunities to discuss research outcomes 
concerning their college or, particularly, other colleges occur in short panel discussions at 
conferences, such as those held by the American Association of Community Colleges 
(AACC) or the League for Innovation in the Community College, or at one-shot faculty 
workshops. Neither of these types of forums lend themselves to the type of thorough 
discussion that can promote a real understanding of outcome data, what lies behind them, 
and what needs to be done to improve outcomes. Colleges and community college 
organizations need to provide more opportunities for faculty and administrators to exploit 
the richness of the quantitative and institutional data that are available to them. 

Research 

The core strategy of the Achieving the Dream initiative is the use of data and analysis to 
identify and remove barriers to increasing student success at community colleges. There 
is a widespread consensus within the initiative and, indeed, throughout the leadership of 
the community college movement in the United States that colleges must be more “data 
driven,” and shift from a “culture of anecdote” to a “culture of evidence.” There is less 
consensus about exactly what such a shift entails and what constitutes “evidence.” While 
many programs are identified as “best practices” and, in some cases, there appears to be a 
consensus about “what works,” this report has argued that a rigorous look at the 
underlying research yields less than definitive conclusions. 

Research on community colleges faces two important barriers. First, the large majority of 
research on higher education concerns four-year colleges. Second, assessing the 
effectiveness of educational initiatives is thwarted by difficult methodological problems. 
Although there is some high-quality research about community colleges and sophisticated 
institutional research departments at some community colleges, in many cases, colleges 
do not have the resources to do this type of research; in other cases, administrators may 
not be convinced that it is necessary. Moreover, the research or studies that have been 
carried out are often published or disseminated in internal reports or other modes that are 
difficult to find, or they are published in such a way that no judgment can be made about 
the accuracy of their conclusions. Clearly there is considerable room for improvement in 
the research basis for “data driven” initiatives to improve community college student 
outcomes. 

(6) It is necessary to recognize that assessments of the effectiveness of practices 
are difficult and involve a continuum of activities and analyses that range 
from simple descriptive comparisons to more time-consuming and expensive 
controlled analyses and experiments. Descriptive comparisons of outcome 
measures for participants and non-participants are a useful beginning to any 
analysis, but more definitive research requires careful attention to the 
characteristics of participants and non-participants, and to the process through 
which students enroll or are recruited into relevant programs. Randomized 
experiments provide some solutions to these problems, but they are expensive and 
difficult to implement. Given the current state of research in this area, a great deal 
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of progress can be made with currently available data or state data that are 
increasingly available. 

(7) Studies must pay increased attention to college-wide changes and the 
institutionalization of promising practices. Research focused on practices to 
improve retention and completion tends to analyze specific programs. These are 
discrete, well-defined activities and it is usually possible to know whether a 
student is enrolled or not. Other innovations, such as organizational changes or 
new hiring and professional development policies, are more difficult to isolate for 
assessment purposes. Moreover, even if discrete practices are successful for a 
small number of students, they may not be easily “brought to scale” or 
institutionalized. Indeed, a successful institution-wide reform may not look 
anything like a discrete program. Researchers, college administrators, and faculty 
need to focus more attention on college-wide innovations and develop methods to 
study them and identify successful strategies. Comparing outcomes before and 
after a change is a first step, but researchers must also take account of other 
factors that might have changed as well. 

( 8 ) Community colleges must recognize the resource needs of institutional 
research. While it is easy to agree with the need to strengthen a culture of 
evidence in principle, it may not seem so obvious if a college president is faced 
with a choice between expanding the developmental education services and 
increasing the institutional research function. Institutional research must be seen 
as an investment that will make other resources more effective. 

(9) More systematic methods to publicize and disseminate useful research 
findings from state and institutional research offices must be developed. State 
and college-level researchers frequently carry out analyses and assessments of 
policies and practices, but often they are not published in ways that make them 
useful to wider audiences. Many reports of “best practices” fail to provide enough 
backup information and data for readers to make a judgment about the relative 
effectiveness of the practice. Assessment results are often only available in 
PowerPoint slides, institutional reports, various types of testimony, or in the files 
of institutional researchers. The type of discussion that would arise from more 
systematic dissemination of local and state research results should be seen as a 
central element of an increased emphasis on a “culture of evidence.” 

(10) Researchers must develop models designed specifically to study community 
colleges. The student integration model has been the most influential perspective 
on student retention. Researchers have recognized, though, that this view is less 
obviously applicable to commuter schools and particularly to community colleges 
that are dominated by part-time students. Indeed, this may be why empirical work 
on the effects of integration at community colleges has been much more 
ambiguous than such work for four-year colleges. Researchers and college faculty 
and administrators need to consider alternative approaches and frameworks. 
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(11) The wide variation in college performance must be exploited to develop 
insights about effective strategies and policies. We have shown that there is a 
wide variation among community colleges in measured cohort graduation rates 
and have developed a benchmarking index that can be used to compare graduation 
rates. As expected, for the most part, these measures are also related to college 
size, demographics, and financial characteristics, although they clearly need to be 
refined and supplemented. Nevertheless, variations in college performance offer 
an as yet unexploited opportunity to examine, using a variety of methodologies, 
the effectiveness of different institutional strategies and characteristics. 

(12) Collaboration between academic, institutional, and state-level researchers 
should be promoted. Researchers working to improve the performance of 
community colleges face formidable problems. They will have more chance of 
success if they use a variety of methodologies and if they combine research based 
on national, state, and local datasets as well as specific institutional and state-level 
knowledge. 

(13) It is crucial to act now, but question and measure. We have been critical of the 
quantity and quality of research on the effects of institutional practices on 
community college graduation and completion rates, although we have found 
high-quality research on community college practices and sophisticated 
institutional research departments. Nevertheless, relevant results of much of the 
research that we have reviewed are either subject to alternative interpretations or 
published in such a way that makes it difficult to evaluate their validity. 

Therefore, its value as a guide for practice and policy must be carefully 
considered. 

College personnel now have a variety of sources of information — including 
experience, institutional knowledge, and research of varying degrees of 
reliability — and this type of information serves as the basis for action. Colleges 
are, of course, ongoing operations and cannot wait for definitive evidence before 
acting; and, in any case, research should not be viewed in absolute terms — either 
definitive or useless. Still, faculty and administrators should move forward with 
the understanding that questions remain about the effectiveness of what they are 
doing. 

Our suggestion is that, in planning activities, colleges should search for the best 
information they can find, but they should search critically, recognizing that all 
research is not the same and that even the most definitive studies, such as those 
using random assignment methodologies, have limitations. At the same time, they 
should do what they can to monitor progress, and do so in a way as thoroughly 
and rigorously as possible. The interaction between research and practice is not a 
search for the definitive answer of “what works.” Rather, it is a constant and 
continuous process and conversation within and among the colleges, and with 
outside researchers and policymakers, as practitioners try to improve their 
practice in the context of a constantly changing environment. We urge that, in this 
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conversation, colleges use the best possible data and the most appropriate 
methodologies. The process of continuous discussion is probably better described 
as a “culture of inquiry” rather than a “culture of evidence.” When speaking of 
monitoring disarmament treaties, Ronald Reagan said, “Trust, but verify.” When 
developing and implementing innovations to improve student outcomes, we say 
“Act now, but question and measure.” 
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APPENDIX: 

Results of Group Logistic Regression on Degree Completion 



MODEL 1 


MODEL 2 


MODEL 3 


Coef (std , , , 

; dy/dx 

err) 


Coef (std , , , 

; dy/dx 

err) 


Coef (std , . , 

\ dy/dx 

err) 



College is located in urban area 


-0.215 


-0.038 


-0.009 


-0.002 


-0.012 


-0.002 




0.044 




0.037 




0.038 




College is located in rural area 


0.184 


0.034 


0.050 


0.009 


0.050 


0.009 




0.092 




0.077 




0.078 




College is a Historically Black College or University 


0.173 


0.032 


0.470 


0.095 


0.486 


0.097 




0.251 




0.209 




0.211 




College is a tribal college 


-0.778 


- 0.111 


-0.863 


-0.122 


-0.753 


-0.108 




0.640 




0.742 




0.747 




1001-2500 FTE undergraduates 






-0.131 


-0.023 


-0.110 


-0.019 








0.073 




0.076 




2501-5000 FTE undergraduates 






-0.485 


-0.082 


-0.455 


-0.076 








0.080 




0.085 




More than 5000 FTE undergraduates 






-0.719 


-0.114 


-0.678 


-0.107 








0.084 




0.091 




Institution awards more certificates than associates 






0.373 


0.071 


0.328 


0.062 








0.057 




0.059 




Percent FTE African-American undergraduates 






-0.906 


-0.163 


-0.922 


-0.165 








0.187 




0.202 




Percent FTE native American undergraduates 






0.203 


0.037 


0.032 


0.006 








0.711 




0.727 




Percent FTE Asian undergraduates 






-1.112 


-0.201 


-1.145 


-0.205 








0.449 




0.487 




Percent FTE Hispanic undergraduates 






-0.486 


-0.088 


-0.530 


-0.095 








0.144 




0.174 




Percent FTE part-time undergraduates 






-0.747 


-0.135 


-0.820 


-0.146 








0.196 




0.212 




Percent FTE female undergraduates 






-2.773 


-0.500 


-2.464 


-0.440 








0.298 




0.331 




Average in-state tuition [$ 1000 ] 










-0.067 


-0.012 












0.037 




Instructional expenditures per FTE undergraduate [$ 1000 ] 










0.069 


0.012 












0.019 




Academic support per FTE undergraduate [$ 1000 ] 










0.031 


0.005 












0.049 




Student services per FTE undergraduate [$ 1000 ] 










-0.080 


-0.014 












0.041 




Federal aid (Pell grants) per FTE undergraduate [$ 1000 ] 










-0.002 


0.000 












0.070 




Constant 


-0.901 




0.984 




0.701 






0.077 




0.210 




0.244 




Adjusted R squared 
Number of observations 


0.352 

884 




0.601 

861 




0.602 

841 




Notes: Bolded coefficients are significant at the 5% level. Regressions also include state dummies (not 

shown). Coefficients for these variables are available from authors on request. 
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